Health-checks.

Health-checks are simple scripts which inform us about status of our service. They are running inside the service's container and report its status to Armada agent. You can see it in 'Status' column of armada list command.

  • passing - everything works!
  • warning - service works, but there's something wrong with it (e.g.: a lot of errors being logged).
  • critical - service doesn't work! Armada won't use its address in any load-balancer until the status changes back to passing or warning.

Default health-check

Every service based on microservice image has a default health-check which checks whether port 80 inside the container is open. If it is, we assume the service works and return 'passing' status. Otherwise 'critical' is returned. You can disable it in Dockerfile with one line:


RUN rm -rf /opt/microservice/health-checks

Custom health-checks

In order to add a health-check to your service, simply create health-checks directory in your project and add your health-checks there. There are some things you should know:

  • shebangs: Since you can write your health check in any language you want, remember to add a shebang to let a system know how to run your script. For example for python health-check you'd add #!/usr/bin/env python.

  • exit codes: Health-checks report their status by exiting with an integer code. 0 means passing, 1 means warning and 2 means critical.

  • the worst outcome: Each health-check script can return a different status. Final service's status is "the worst" of all statuses. So if you have 10 health checks, 9 of them exit with code 0 (passing), and one exits with code 2 (critical) your service's final status will be critical.

Example

Instead of providing some dummy example, we suggest you check out armada mysql repository to see how custom health-checks can be used in real life.
As you can see, we're logging "Error. MySQL is not running." information. This is message will be visible in service's health-checks logs, which, in case of mysql service, you can see either by running armada diagnose mysql command, or by reading health-checks logs file: armada ssh mysql tail /var/log/supervisor/run_health_checks*