Monitor Amazon OpenSearch Service configuration modifications extra simply with new visibility enhancements

Amazon OpenSearch Service presents a number of area configuration settings to satisfy your workload-specific necessities. As a part of customary service operations, you might be required to replace these configuration settings frequently. Just lately, Amazon OpenSearch Service launched visibility enhancements that permit you to monitor configuration modifications extra successfully. We’ve launched granular and extra descriptive configuration statuses that allow you to arrange alarms and use them in automation to reduce handbook monitoring.

We advocate that you just make the most of these visibility enhancements in your purposes. These modifications are backward suitable, and in case your automations depend on the legacy processing parameter to find out configuration change standing, then they need to nonetheless proceed to work with none disruption. To simplify monitoring of a number of in-flight configuration change requests, Amazon OpenSearch Service permits configuration request solely when Area Processing Standing is Energetic. Extra particulars are in part ‘Single configuration change at a time’.

Resolution overview

Earlier, configuration change standing visibility was accessible by means of processing parameters within the OpenSearch Service APIs (Software Programming Interface), and as a Area Standing discipline within the OpenSearch Service console. We’ve got now launched the next modifications to enhance the configuration replace expertise:

• Launched two new parameters, DomainProcessingStatus and ConfigChangeStatus, within the API responses. Equally, added Area Processing Standing and Configuration Change Standing fields within the console. These modifications present higher visibility by means of a number of, intuitive statuses. Earlier statuses had been restricted to solely two values: Energetic and Processing.
• Capacity to simply evaluate energetic and in-flight configurations for readability. Earlier, it required a number of steps.
• Amazon OpenSearch Service has now adopted the method of permitting a single configuration change request at a time. There isn’t a restrict on the variety of area configuration modifications you possibly can bundle in a single request. Nonetheless, you possibly can submit the following configuration request when the earlier request is full and the area processing standing turns into Energetic. This enchancment streamlines configuration updates and addresses earlier challenges of monitoring a number of, in-flight configuration change requests.
• Capacity to cancel a change request in case of a validation failure. Beforehand, when situations had been unavailable, domains remained in processing state. Now, upon encountering any validation failure, you possibly can cancel the change request and retry after a while.
• Area processing standing turns to Energetic solely after all of the background actions, together with shard motion is full. This implies that you could confidently use newly launched statuses in your automation scripts without having to deduce if all the inner processes, corresponding to information motion, are full.

How do you get granular particulars to trace the configuration replace standing?

As a part of latest enhancements, Amazon OpenSearch Service launched DomainProcessingStatus and ConfigChangeStatus parameters within the APIs together with the respective Area Processing Standing and Configuration Change Standing fields within the console. You’ll be able to depend on these statuses to get correct and constant data throughout totally different configuration change eventualities, like when configuration modifications contain blue/inexperienced operations or with out blue/inexperienced operations, and when configuration modifications are triggered by the operator or by the OpenSearch Service. Allow us to discover these enhanced visibility experiences.

1. Area processing standing visibility: You’ll be able to monitor the staus of domain-level configuration modifications by means of the Area Processing Standing discipline within the console. Equally, API responses embrace the DomainProcessingStatus parameter. The values and a quick description are supplied within the following particulars:
   1. Energetic: No configuration change is in progress. You’ll be able to submit a brand new configuration change request.
   2. Creating: New area creation is in progress.
   3. Modifying: This standing signifies that a number of configuration modifications, corresponding to new information node addition, Amazon Elastic Block Retailer (Amazon EBS) GP3 storage provisioning, or establishing KMS keys, are in progress. In different phrases, modifications made by means of the UpdateDomainConfig API, set the standing to modifying. The ‘Modifying’ standing additionally covers conditions the place domains require shard motion to finish configuration modifications. Be aware: For backward compatibility, we have now stored the habits of the processing parameter unchanged within the API responses, and it’s set to false as quickly because the core configuration modifications are full, with out ready for shard motion completion.
   4. Upgrading Engine Model: Engine model upgrades are in progress, corresponding to from Elasticsearch model 7.9 to OpenSearch model 1.0.
   5. Updating Service Software program: This standing refers to configuration modifications associated to service software program updates.
   6. Deleting: Area deletion is progressing.
   7. Remoted: This represents domains which might be suspended on account of quite a lot of causes, corresponding to account-related billing points or domains that aren’t compliant with vital safety patch updates.
2. Configuration change standing visibility: Configuration modifications will be initiated by the consumer (e.g., new information node addition, occasion sort change) or by the Service (e.g., AutoTune and obligatory service software program updates). You’ll find the newest standing particulars by means of Configuration Change Standing discipline within the console, and thru the ConfigChangeStatus parameter in API responses. Beneath are the values and a quick description:
   1. Pending: Signifies {that a} configuration change request has been submitted.
   2. Initializing: Service is initializing a configuration change request.
   3. Validating: Service is validating the requested modifications and sources required.
   4. Validation Failed: Requested modifications failed validation. At this level, no configuration modifications are utilized. Some potential validation failures might be the presence of purple indices within the area, unavailability of a selected occasion sort, and low disk house. Right here is a listing of potential validation failures. Throughout a validation failure occasion, you possibly can cancel, retry, or edit configuration modifications.
   5. Awaiting consumer inputs: Eventualities the place consumer might be able to repair validation errors corresponding to invalid KMS key. At this standing, consumer can edit the configuration modifications.
   6. Making use of modifications: Service is making use of requested configuration modifications.
   7. Cancelled: Throughout validation failed standing, you possibly can both click on on the Cancel button within the console or name the CancelDomainConfigChange API. All of the utilized modifications that had been a part of the change request will probably be rolled again.
   8. Accomplished: Requested configuration modifications have been efficiently accomplished.

Console enhancements

The Amazon OpenSearch Service console presents enhanced visibility to trace configuration change progress. Beneath are just a few screenshots to present you an thought of those enhancements.

• Amazon OpenSearch Service console gives Area Processing Standing, Configuration Change Standing, and Change ID fields. Be aware: To know the change particulars related to the Change ID, you need to use the DescribeDomainChangeProgress API.

• Configuration change abstract. To see a facet by facet comparability of your energetic configurations and requested modifications, on the area element web page, navigate to the cluster configuration tab, scroll all the way down to the configuration change abstract part. Pending Adjustments discipline exhibits the standing of the pending properties at the moment and doesn’t embrace modifications which have been utilized. You may also get comparable particulars from the DescribeDomain and DescribeDomainConfig APIs by means of theModifyingProperties parameter.

Cancelling throughout validation failure. Within the beneath screenshots, you possibly can see a brand new choice to cancel a change request when a configuration change request fails validations. For instance, once you encounter SubnetNotFound error, you need to use the Cancel request button to roll again to the earlier energetic configuration, repair the problem after which retry the configuration replace.

Single configuration change at a time

Beforehand, it was not simple to trace the success and failure of particular person change requests, when a number of requests had been made. To supply a simplified expertise, OpenSearch Service now limits you to solely a single change request at a time. In a single configuration change request, you possibly can bundle a number of modifications without delay. As soon as a configuration change request is submitted, it have to be accomplished earlier than you possibly can request the following configuration change by means of the console, or by means of the UpdateDomainConfig API. This simplified expertise makes it simpler to maintain monitor of modifications which have been requested and their most up-to-date standing. In case your automation is written to name configuration change replace APIs a number of instances, then it ought to be up to date to group a number of configuration modifications in a single replace name, or await particular person updates to finish earlier than you submit the following configuration change. You’ll be able to replace area configuration when area processing standing turns into energetic. For a listing of modifications which may want a blue/inexperienced deployment, please see right here.

The beneath screenshot exhibits an instance alert on the ‘Edit area’ web page informing the consumer that one other change or replace is in progress. OpenSearch Service not lets you submit new configuration replace requests, and the ‘Apply change’ button is disabled till the change in progress is accomplished.

API modifications

You should utilize the DescribeDomain, DescribeDomainChangeProgress, and DescribeDomainConfig APIs to get detailed configuration replace statuses. As well as, you need to use CancelDomainConfigChange to cancel the change request within the occasion of a validation failures. You’ll be able to refer Amazon OpenSearch Service API documentation right here.

Conclusion

On this publish, we confirmed you how one can get granular details about a configuration replace request. These newly launched modifications will enable you achieve higher visibility into the progress of configuration change requests, and simply distinguish between utilized modifications and pending ones. You want to be sure that the DomainProcessingStatus processing standing worth is Energetic earlier than submitting configuration change requests. The power to cancel modifications within the occasion of validation failures offers you higher management in getting your area out of processing state in a self-service method. Go to product documentation to study extra.

Concerning the Authors

Siddhant Gupta is a Sr. Technical Product Supervisor at Amazon Net Companies primarily based in Hyderabad, India. Siddhant has been with Amazon for over six years and is at the moment working with the OpenSearch Service crew, serving to with new area launches, pricing technique, and bringing EC2 and EBS improvements to OpenSearch Service prospects. He’s obsessed with analytics and machine studying. In his free time, he loves touring, health actions, spending time along with his household and studying non-fiction books.

Deniz Ercelebi is a Sr. UX Designer at Amazon OpenSearch Service. In her function she contributes to the creation, implementation, and profitable supply of design options for complicated issues. Her private drive is fueled by a ardour for consumer expertise, a dedication to customer-centric options, and a agency perception in collaborative innovation.

Shashank Gupta is a Sr. Software program Developer at Amazon OpenSearch Service, specializing within the enhancement of the Managed service side of the platform. His major focus is on optimizing the managed expertise, spanning from the console to APIs and useful resource provisioning in an environment friendly method. With a devoted dedication to innovation, Shashank goals to raise the general buyer expertise by introducing ingenious options inside the service.