WordPress on Linux App Service in Azure and nginx.conf troubles

I knew that I would be facing some annoyances due to the nature of the new Azure Linux App Service on which this blog is now running, even though the migration process went smooth overall. I did obsess on a multitude of minor details for a time, though most of them were easily solved by some tinkering with the theme files. Another set of annoyances required some changes in the nginx config files, and while those are easily solved too, the real issue is getting the changes to persist.

Like other container-based solutions, the WordPress on Linux App Service starts fresh every time the container is recycled, and resets a horde of settings to their defaults. That’s usually not a big issue, as most solutions have some way of working around this. Most solutions are not Azure-based though, and haven’t suffered the light touch of Microsoft’s dev teams. And after wasting a lot of time to address what should have been a 5-minute task, it’s time for another rant. Don’t worry, I will also present a solution 🙂

Anyway, so how do we go about persisting configuration changes, such as those made to nginx’s default.conf? You can go in two directions about this – mount an external volume and map it to /etc/nginx, or run a startup script to overwrite the files located in said directory. The first method was what a friend of mine, expert in all things Linux and containers recommended, yet at the end the solution didn’t work – the files were being overwritten with the default values. So, I opted out for the second solution, configuring a startup script. This looks to be the preferred method for Azure App Services, judging by the presence of the Startup Command setting under the App Service > Configuration > General settings page. Yet, after a gazillion of variations on what should’ve been a very simple script, I was getting nowhere close to a solution.

After a long, long time spent in needless troubleshooting, I finally run into the following statement:

Note: As of September 2022, the startup script MUST exist at (/home/dev/startup.sh). Further, the value of the Startup Command field in the Azure Portal found under [App Service for Linux] > Configuration > General Settings > Startup Command is ignored. The Azure team is aware of this and working to make this flow more natural.

What! the! actual! fck! You would think that a detail as important as this one would have made it to the stupid UI, or at the very least the official documentation. Yet, the only place I saw this mentioned was the GitHub repo of the WordPress on Linux App Service project. Sure, it’s a logical place to find instruction on, but a lot more logical would be to have this mentioned in the official documentation (totally useless in this scenario) to which the Learn more link under the Startup command control points. Oh well, that’s Microsoft’s documentation for you.

Now that the rant is over, let’s summarize the actual solution. Well, let me present the problem I was trying to solve first. In a nutshell, I wanted to make some changes to the nginx config files in order to enable few things:

• Some legacy redirects left from the time I was running my blog on SharePoint Online
• Blocking some known abusers via $http_user_agent match
• Fixing favicon.ico annoyances
• Most importantly, making sure WordPress (i.e. the “backend”) gets to see the proper client IP

The last issue required some additional research, and was again not helped by the lack of proper documentation on Microsoft’s side. Bit by bit I managed to piece the whole picture and find a proper solution, which in hindsight turned to be quite simple. All you need to do is add two lines in the server block of your nginx.conf file, which instruct it to use the “real” IP as provided by the ngx_http_realip_module module. The module itself comes preinstalled on the container, so all you have to do is get the local ip and instruct the module to “translate” it:

set_real_ip_from 169.254.129.0/24;
real_ip_header X-Forwarded-For;

While this solved the issue, my troubles were unfortunately not over. Dure to the aforementioned behavior of the container, I had to make sure the changes to nginx.conf are not being overwritten on recycle, and here again I run into some shortcomings. While the solution based on the /home/dev/startup.sh script works, it does so only temporary as 3-4 minutes after the container is started, another process seems to be overwriting the nginx.conf file with its default values. And on top of that, it looks like the script itself was getting executed multiple times, much like reported in this GitHub issue. Yay!

After yet another few hours spent in research, annoying my friend and trying various solutions, we ended up with a simple, albeit quite unexpected one – a sleep 300 entry on top of the startup.sh file seems to do the trick. Hardly an elegant solution, and certainly not one without its own troubles, but at this point I was willing to give up altogether.  I can live with the potential implications of running the site with the default nginx.conf for the first five minutes after restart, as long as the required changes do get eventually stamped on it. And who knows, the Azure folks might actually get their game together in the future, and make such workarounds unneeded.

sleep 300
cp /home/dev/default.conf /etc/nginx/conf.d/default.conf
/usr/sbin/nginx -s reload

Right, that’s it for now. I hope my findings help someone, share the pain people. I did enter some additional details on the IP issue over at this Q&A thread.