SAPHanaSR-manageAttr(8)

NAME
SAPHanaSR-manageAttr - Helps managing Linux cluster attributes for SAP HANA system replication.

SYNOPSYS
SAPHanaSR-manageAttr [ --help | --version ]
SAPHanaSR-manageAttr --sid=SID --ino=INSTNO --case usecase [ --check | --migrate resource ]

DESCRIPTION
SAPHanaSR-manageAttr helps managing Linux cluster attributes for SAP HANA system replication.

Current sole use case is the upgrade of RAs and HADR provider hook script to "multi-target". Defined entry and target states, upgrade procedures and requirements are described in the EXAMPLES and REQUIREMENTS sections below.

OPTIONS

--help
    show help

--version
    show script version

--check
    check, if all recommended pre-requisites for an upgrade are met

--migrate resource
    migrate that given resource, related to the use case

--case usecase
    use case to process, currently only upgrade to "multi-target"

--sid=SID
    use SID of the SAP system

--ino=INSTNO
    use instance number of the SAP system

SUPPORTED PARAMETERS

hana_${sid}_gra = [ generation ]
    The resource agent (RA) generation attribute identifies which generation of the RA is running. It helps determing RA's capabilities and performing cluster-wide upgrades of RA and srHook. The generation should be same for both on all nodes of the Linux cluster after successful upgrade. See also hana_${sid}_gsh.

hana_${sid}_gsh = [ generation ]
    The srHook generation attribute identifies which generation of the srHook is running. It helps determing srHook's capabilities and performing cluster-wide upgrades of RA and srHook. E.g. starting with generation 2.0 the RA supports scale-out multi-target system replication, which needs replacement of the old SAPHanaSR.py by new SAPHanaSrMultiTarget.py. See also hana_${sid}_gra.

hana_${sid}_site_srHook_${SITE} = [ SOK | SFAIL | SWAIT | SREG | PRIM ]
    The SAPHanaController and SAPHana RA recognize the effecting system replication status via srHook attributes in the CIB properties section. The scale-out multi-target aware attribute is site-specific. This attribute is used after successful upgrade or fresh installation into target state. See also SAPHanaSrMultiTarget.py(7). The SAP System Indentifier (sid) is given in lower case.

hana_${sid}_glob_srHook = [ SOK | SFAIL | SWAIT | SREG | PRIM ]
    The old-style srHook attributes in the CIB properties section does not contain site-specific information. This attribute is to be replaced by the upgrade. See also SAPHanaSR.py(7). The SAP System Indentifier (sid) is given in lower case.

hana_${sid}_glob_mts = [ true | false ]
    The multi-target support attribute identifies whether the Linux cluster supports HANA scale-out multi-target system replication. The SAP System Indentifier (sid) is given in lower case.

hana_${sid}_glob_upd = [ ok | nok ]
    The update status attribute identifies whether the whole cluster has successfully passed an update procedure. The SAP System Indentifier (sid) is given in lower case.

RETURN CODES
0 Successful program execution.
>0 Usage, syntax or execution errors.

EXAMPLES
* Overview on upgrading old-style hook to multi-target hook. Upgrade procedure from defined entry state with old-style hook into defined target state with multi-target enabled hook. This case is the main target for the upgrade. Certain steps are described in more detail as separate examples. Also see REQUIREMENTS section below for defined states. The procedure at a glance:
a. Initially check if everything looks fine.
b. Set Linux cluster resources SAPHanaController and SAPHanaTopology into maintenance.
c. Install multi-target aware SAPHanaSR-ScaleOut package on all nodes.
d. Adapt sudoers permission on all nodes.
e. Replace HANA HADR provider configuration on both sites.
g. Check Linux cluster and HANA HADR provider for matching defined upgrade entry state.
h. Migrate srHook attribute from old-style to multi-target.
i. Check Linux cluster for matching defined upgrade target state.
j. Set Linux cluster resources SAPHanaController and SAPHanaTopology from maintenance to managed.
k. Optionally connect third HANA site via system replication outside of the Linux cluster.
l. Finally check if everything looks fine. As final result of this procedure, the RAs and hook script are upgraded from old-style to multi-target. Further the Linux cluster's old-style global srHook attribute hana_${sid}_glob_srHook is replaced by site-aware attributes hana_${sid}_site_srHook_${SITE}. The new attributes might stay empty until HANA raises srConnectionChanged() events and triggers the new hook script for the first time. Alternatively the new attribute values can be set manually. The HANA global configuration and Linux sudoers permissions are adapted. New auxiliary Linux cluster attributes are introduced, see PARAMETERS section. * Overview on upgrading existing multi-target hook to new one. Upgrade procedure from defined entry state already using a specific multi-target aware hook into defined target state with general product multi-target aware hook. This case is relevant for early adopters who attended the ramp-up phase. Certain steps are described in more detail as separate examples. Also see REQUIREMENTS section below for defined states. The procedure at a glance:
a. Initially check if everything looks fine.
b. Set Linux cluster resources SAPHanaController and SAPHanaTopology into maintenance.
c. Install multi-target aware SAPHanaSR-ScaleOut package on all nodes.
d. Check Linux cluster for matching defined upgrade entry state.
e. Reload HANA HADR provider hook script on both sites.
f. Check Linux cluster for matching defined upgrade target state.
g. Set Linux cluster resources SAPHanaController and SAPHanaTopology from maintenance to managed.
h. Finally check if everything looks fine. As final result of this upgrade, the already multi-target aware RAs and hook script are updated to an new version. The existing site-aware attributes hana_${sid}_site_srHook_${SITE} are kept. New auxiliary attributes are introduced, see PARAMETERS section. * Overview on upgrading old-style hook, no multi-target hook needed. Upgrade procedure from defined entry state with old-style hook into defined target state while keeping old-style hook. This case is relevant for customer who do not want to change from old-style to multi-target, but just upgrade the RPM. Certain steps are described in more detail as separate examples. Also see REQUIREMENTS section below for defined states. The procedure at a glance:
a. Initially check if everything looks fine.
b. Set Linux cluster resources SAPHanaController and SAPHanaTopology into maintenance.
c. Install multi-target aware SAPHanaSR-ScaleOut package on all nodes.
d. Reload HANA HADR provider hook script on both sites.
e. Check Linux cluster for matching defined upgrade target state.
f. Set Linux cluster resources SAPHanaController and SAPHanaTopology from maintenance to managed.
g. Finally check if everything looks fine. As final result of this upgrade, the RAs and hook script are updated to an new version. The existing old-style global attribute hana_${sid}_glob_srHook is kept. New auxiliary attributes are introduced, see PARAMETERS section. * Overview on downgrade from multi-target hook to old-style hook. Unsupported downgrade procedure from defined entry state with multi-target hook back into defined target state with old-style hook. This might be useful when repeating tests for an upgrade procedure. All downgrade steps and checks need to be done manually. The procedure at a glance:
a. Initially check if everything looks fine.
b. Disconnect third system replication HANA from the two sites inside the Linux cluster.
c. Set Linux cluster resources SAPHanaController and SAPHanaTopology into maintenance.
d. Downgrade SAPHanaSR-ScaleOut package on all nodes.
e. Adapt sudoers permission on all nodes.
f. Replace HANA HADR provider configuration on both sites.
g. Remove multi-target attributes from CIB.
h. Add old-style attributes into CIB.
i. Check Linux cluster for matching defined downgrade target state.
j. Set Linux cluster resources SAPHanaController and SAPHanaTopology from maintenance to managed.
l. Finally check if everything looks fine. As final result of this procedure, the RAs and hook script are downgraded from multi-target to old-style. Further the Linux cluster's multi-target attributes are removed and old-style attributes are added, particularly the global srHook. The HANA global configuration and Linux sudoers permissions are adapted. * Overview on upgrading the SAPHanaSR-ScaleOut package. This procedure can be used to upgrade RAs, HANA HADR provider hook scripts and related tools while HANA and Linux cluster are staying online. For details see SAPHanaSR_maintenance_examples(7). The procedure at a glance:
1. Check status of Linux cluster and HANA.
2. Set resources SAPHanaController and SAPHanaTopology to maintenance.
3. Update SAPHanaSR-ScaleOut RPM on all cluster nodes.
4. Refresh resources SAPHanaController and SAPHanaTopology.
5. Optionally set resources SAPHanaController and SAPHanaTopology out of maintenance.
6. Check status of Linux cluster and HANA. * Check Linux cluster for matching defined upgrade state. This should be always the first step during upgrade procedure, after having the RPM upgraded. The check shows if all nodes are online, no resource failures are recorded, HANA is healthy and system replication is in sync. The whole Linux cluster is checked at once. The resources SAPHanaController and SAPHanaTopology have to be in maintenance mode. Of course the same check is useful after the upgrade has been completed. See also SAPHanaSR_maintenance_examples(7). Example SID is ABR, instance number is 00.

    # crm_mon -1r
    # cs_clusterstate -i
    # SAPHanaSR-manageAttr --sid=ABR --ino=00 --case multi-target --check
    # SAPHanaSR-showAttr
* Check HANA HADR provider configuration and related script. Manual checks can be done in addition to the built-in checks of SAPHanaSR-manageAttr. The check should be done at both sites. See below examples on HANA HADR provider configuration and hook script, sudoers permission, the FILES section, and SAPHanaSrMultiTarget.py(7) for details. * Remove pretty ornaments from script output. This might be useful when capturing script output for documentation, or if screen width is limited. Example SID is ABR, instance number is 00.

    # SAPHanaSR-manageAttr --sid=ABR --ino=00 --case multi-target --check 2>&1 | colrm 1 6
* Add sudoers permission for multi-target HADR provider hook script. This needs to be done before the new multi-target HADR provider is loaded. Old and new permission can be set in parallel during migration phase. First the respective file is identified, second the new entries are added, last the effective permissions are displayed. Of course the old permission should be removed once the migration has been completed.
In this example the SID is ABR - thus lowercase-sid abr, the respective file is /etc/sudoers.d/SAPHanaSR, a broad new permission is added for simplicity.
See SAPHanaSR.py.(7), SAPHanaSrMultiTarget.py(7) and sudo(8) for details and more secure permissions. For removing the old permission, see example below.

    # grep "NOPASSWD.*crm_attribute.*hana_" /etc/sudoers /etc/sudoers.d/*
    # cp /etc/sudoers.d/SAPHanaSR /etc/sudoers.d/SAPHanaSR.BAK
    # echo "abradm ALL=(ALL) NOPASSWD: /usr/sbin/crm_attribute -n hana_abr_site_*" >/etc/sudoers.d/SAPHanaSR
    # sudo -U abradm -l | grep "NOPASSWD.*crm_attribute.*hana_"
* Remove sudoers permission for old-style HADR provider hook script. This should to be done after the new multi-target HADR provider has been loaded. First the respective file is identified, second the old entries are deleted, last the effective permissions are displayed. Of course the new multi-target permission needs to be added before the new HADR provider is loaded.
In this example the SID is ABR - thus lowercase-sid abr, the respective file is /etc/sudoers.d/SAPHanaSR, the permission is just made in-active. Of course the line can be entirely removed also.
See SAPHanaSR.py.(7), SAPHanaSrMultiTarget.py(7) and sudo(8) for details. For adding the multi-target permission, see example above.

    # grep "NOPASSWD.*crm_attribute.*hana_" /etc/sudoers /etc/sudoers.d/*
    # visudo -f /etc/sudoers
    ## old-style permission is not needed anymore
    ## abradm ALL=(ALL) NOPASSWD: /usr/sbin/crm_attribute -n hana_abr_glob_srHook_*
    # sudo -U abradm -l | grep "NOPASSWD.*crm_attribute.*hana_"
* Replace HANA HADR provider configuration. The Linux cluster and HANA have to be checked for matching the entry state prior to replacing the HANA HADR provider configuration. The section [ha_dr_provider_saphanasr] needs to be replaced by the new section [ha_dr_provider_saphanasrmultitarget]. The respective line of the section [trace] should be adapted as well. This task needs to be done at both sites. Since old and new attributes should not be mixed, changes on both sites might be done together. To make changes active, HANA needs to be informed about the change. This can be done by re-loading the hook as shown in another example. Do not forget to adapt the sudoers permission to match the new attribute. See also SAPHanaSrMultiTarget.py(7) and REQUIREMENTS section below. Example lowercase-SID is abr.
The global.ini should be handled by SAP tools like HANA Cockpit. This example on changing the file directly is for test clusters only. It should not be applied on production systems.

    # su - abradm
    ~> cdcoc
    ~> cp global.ini global.ini.BAK
    ~> sed -e s/saphanasr/saphanasrmultitarget/g -e s/\ SAPHanaSR$/\ SAPHanaSrMultiTarget/g <global.ini.BAK >global.ini.NEW
    ~> cp global.ini.NEW global.ini
    ~> cat global.ini
    ~> exit
* Reload hook script from persistence into running HANA. When installing an new package version, the HANA HADR provider hook script might be updated on-disk. Nevertheless HANA is still running with the old version of that script. To make changes active, the new script can be loaded from disk into HANA. This should be done on both sites. Please refer to SAP documentation for details. Example lowercase-SID (sid) is abr.

    # su - abradm
    ~> hdbnsutil -reloadHADRProviders; echo $?
    ~> cdtrace
    ~> grep HADR.*load.*SAPHanaS nameserver_*.trc
    ~> exit
* Upgrade srHook attribute from old-style to multi-target. The Linux cluster and HANA have to be checked for matching the entry state prior to migrating the srHook attribute. The new attributes might stay empty until HANA raises srConnectionChanged() events and triggers the new hook script for the first time. The correct hook script needs to be already configured in HANA's global.ini. The correct sudoers permission need to be set already. See example above and REQUIREMENTS section below and also SAPHanaSR_maintenance_examples(7). Example SID is ABR, lowercase-SID is abr, instance number is 00, SAPHanaController m/s resource is msl_SAPHanaCon_ABR_HDB00.

    # SAPHanaSR-showAttr
    # SAPHanaSR-manageAttr --sid=ABR --ino=00 --case multi-target --migrate msl_SAPHanaCon_ABR_HDB00
    # SAPHanaSR-showAttr
* Manually enable srHook attribute upgrade. This should not be necessary under normal circumstances. This usually is been done by the SAPHanaSrMultiTarget.py hook script. Example lowercase-SID is abr.

    # crm_attribute -n hana_abr_glob_mts -s SAPHanaSR -t crm_config -v true
* Delete srHook generation (gsh) and RA generation (gra) from node. Might be useful for repeating an upgrade test. Usually this is not needed. Example node is suse12, lowercase-SID (sid) is abr.

    # SAPHanaSR-showAttr
    # crm_attribute --delete -t node -N suse12 -n hana_abr_gsh -l reboot
    # crm_attribute --delete -t node -N suse12 -n hana_abr_gra -l forever
    # SAPHanaSR-showAttr

FILES

/usr/sbin/SAPHanaSR-manageAttr
    the program itself

/usr/share/SAPHanaSR-ScaleOut/SAPHanaSR.py
    the scale-out old-style hook provider, delivered with the RPM for backward compatibility

/usr/share/SAPHanaSR-ScaleOut/SAPHanaSrMultiTarget.py
    the scale-out multi-target aware hook provider, delivered with the RPM

/hana/shared/$SID/global/hdb/custom/config/global.ini
    the on-disk representation of HANA global system configuration

/etc/sudoers, /etc/sudoers.d/SAPHanaSR
    the sudo permissions configuration

/dev/stderr
    SAPHanaSR-manageAttr writes warnings and errors to stderr

REQUIREMENTS
* For upgrading resource agents, hook script and related attributes, Linux cluster and HANA are in one of the defined upgrade entry states. Defined entry states are:

    1. The cluster is using old-style global srHook status attribute. All cluster nodes are online in the cluster and there are no current errors in the cluster or HANA. No third HANA site is attached. Main use case for SAPHanaSR-manageAttr.

    2. The site-based srHook attributes are already in correct use, no old-style global attribute is in use. A third HANA site might be attached. Can be found at early adopters.

    3. The cluster is set up from scratch. A third HANA site might be attached. There are neither old-style global, nor site-based srHook attributes.

* The SAPHanaController m/s resource needs to be set into maintenance mode during upgrade.

* The old-style global srHook status attribute will be deleted as part of the upgrade procedure.

* Both HANA sites need to reload the global.ini and the HADR provider script. If that is achieved by restarting HANA, an SR take-over might help reducing service impact. Please refer to SAPHanaSR_maintenance_examples(7) for how performing an SR take-over.

* The HANA config file global.ini is located at /hana/shared/$SID/global/hdb/custom/config/global.ini .

* The Linux cluster can be either upgraded to the defined upgrade target state, or run unchanged with the old-style global attribute and related hook script. Not allowed is mixing old and new attributes or hook scripts within one Linux cluster.

* Reverting back from new to old setup is not supported at the moment.

* Any procedure has to be tested for matching particular purpose and environment before applying it to production systems.

* See also the REQUIREMENTS section of SAPHanaSR-ScaleOut(7) and SAPHanaSrMultiTarget.py(7) for general requirements and for technical details.

BUGS
* Formatting and content of this script's output will change, since this script is under development. This script is not intended to be called from tools.

* In case of any problem, please use your favourite SAP support process to open a request for the component BC-OP-LNX-SUSE. Please report any other feedback and suggestions to feedback@suse.com.

SEE ALSO
ocf_suse_SAPHanaController(7) , SAPHanaSR-ScaleOut(7) , SAPHanaSR-showAttr(8) , SAPHanaSR_maintenance_examples(7) , SAPHanaSR.py(7) , SAPHanaSrMultiTarget.py(7) , crm_simulate(8) , cibadmin(8) , crm_mon(8) , cs_convert_time(8) , cs_clusterstate(8) , cs_show_hana_info(8) , sudo(8) , sudoers(5) ,
https://documentation.suse.com/sbp/all/?context=sles-sap ,
https://documentation.suse.com/sles-sap/ ,
https://www.susecon.com/archive-2020.html

AUTHORS
A.Briel, F.Herschel, L.Pinne

COPYRIGHT
(c) 2021 SUSE LLC
SAPHanaSR-manageAttr comes with ABSOLUTELY NO WARRANTY.
For details see the GNU General Public License at http://www.gnu.org/licenses/gpl.html