With the increasing popularity of Docker as a containerization platform, it is important for developers to understand how to properly shut down their containers to avoid potential data loss or corruption. In this article, we will discuss the importance of handling graceful shutdown of your Docker Cron containers and how to handle signals.
Why is it important to handle signals
It is important to handle signals in a Docker container because, when you or your orchestrator stop or restart your Docker container, the processes running inside the container will be abruptly terminated unless you have specifically programmed your application to handle signals. This can cause issues with your scheduled cron jobs and lead to data loss or corruption.
In my company, we are running our production workload on AWS Fargate, a compute service provided by Amazon Web Services that allows us to run containers without having to manage the underlying infrastructure. Fargate manages the availability and scalability of the underlying infrastructure, so we can focus on building and deploying our applications.
To reduce our cost, we use AWS Fargate Spot containers to save up to 70% of the costs of a Fargate non-Spot container. AWS Fargate Spot is a cost-effective alternative to the standard Fargate service but your workload needs to be interruption tolerant. Using AWS Fargate Spot comes with a huge constraint : at any time, AWS decides that our container must stop within 120 seconds.
Fargate Spot : SIGTERM signal
How does AWS tell our container to stop? By using Signals ! AWS will send a SIGTERM signal to our container, wait stopTimeout, a time duration to wait before the container is forcefully killed if it doesn’t exit normally on its own :
By properly handling the SIGTERM signal, you can ensure that your cron jobs are completed before the container is stopped, which can help prevent unexpected behavior and maintain the integrity of your application.
The crontab problem
The classic crontab implementation found on most Linux operating systems is not made for a containerized environment. In fact, the classic crontab don’t respond gracefully to SIGINT / SIGTERM and can lead to processes not shutting down cleanly, which can result in resources not being released, data not being saved, or other problems. This makes sense in a server environment where init will handle the orphan jobs, but it’s inappropriate in a container environment because when the process defined in CMD / ENTRYPOINT will die, the whole container die and orphan jobs are killed.
Supercronic to save you
To solve the classic crontab problem for containerized environments, I use Supercronic. This is a tool written in Go whose purpose is to behave exactly how you would expect cron running in a container to behave : SIGTERM triggers a graceful shutdown 🤝
Let’s implement it for a PHP Crontab by creating a Dockerfile :
FROM php:8.2
# Install and enable PCNTL to allow php processes to handle signals
RUN docker-php-ext-install pcntl && docker-php-ext-enable pcntl
# Get supercronic
ENV SUPERCRONIC_URL=https://github.com/aptible/supercronic/releases/download/v0.2.1/supercronic-linux-386 \
SUPERCRONIC=supercronic-linux-386 \
SUPERCRONIC_SHA1SUM=0a0ce84242a56cfc7dcbe0b498160c8109b6d323
# Install supercronic
RUN curl -fsSLO "$SUPERCRONIC_URL" \
&& echo "${SUPERCRONIC_SHA1SUM} ${SUPERCRONIC}" | sha1sum -c - \
&& chmod +x "$SUPERCRONIC" \
&& mv "$SUPERCRONIC" "/usr/local/bin/${SUPERCRONIC}" \
&& ln -s "/usr/local/bin/${SUPERCRONIC}" /usr/local/bin/supercronic
# Get our crontab
COPY ./docker/cron /etc/cron.d/cron
RUN chmod 0644 /etc/cron.d/cron
# Run supercronic
CMD ["/usr/local/bin/supercronic", "-overlapping", "/etc/cron.d/cron"]Conclusion
In conclusion, handling signals in Docker cron containers is important for a number of reasons and it can be done with Supercronic. Firstly, it ensures that the container can gracefully shutdown when needed, allowing for a clean and orderly shutdown process. This can be particularly important when running critical or time-sensitive tasks within the container. Secondly, handling signals can also help to improve the reliability and stability of the container, as it allows the container to properly respond to and recover from any issues that may arise. Overall, taking the time to properly handle signals in Docker cron containers can help to ensure that they are able to run smoothly and efficiently, making them a valuable tool for any application or workload.