In my case, the solution is to update
I got this same issue today and this is How I solved this.
First of all I stopped mysql:
service mysql stop.
After stopping mysql I ran MYSQLD with upgrade:
/usr/sbin/mysqld --upgrade=minimalMake sure upgrade argument is minimal. [https://serverfault.com/questions/1130794/mysql-stopped-working-on-ubuntu-22-04-failed-to-upgrade-server-error]
During my coffee break just a few minutes after lunch, I randomly check my blog post for inspiration.
However, I immediately see that my blog isn't operational;
502 Bad Gateway error.
My first reaction is that my blog is down because I may have missed a monthly payment. I promptly go to Digital Ocean and make a payment.
I restart my server - wait a few minutes - still the same error.
As I need to continue work, I postpone any research for later.
The following day, I glance at my blog again; the same error. A bit confused, I start to investigate further as I haven't updated anything.
I log into my server and use
ghost cli utility to understand the issue. Ghost Doctor indicates that everything else is fine apart from my database connection.
Using the command
sudo service mysql status, I see that indeed my mysql service is stuck with the "Upgrade in progress" message. At this point, I think that this is normal and that I should wait. Yet, after hours, it's still stuck.
What I attempted
I restart the mysql service using
sudo service mysql restart. Nevertheless, this is not possible as there's an
socket lock error. Looking online, I see that this is a common issue and there are many solutions. (way too many actually)
The first answer I read on StackExchange seems relevant to this particular issue
MySQL Update in Progress and looks promising. Nevertheless, in the end, the commenter describes the process of backing up the database and restoring it. For me, this looks like too much work and I skip it. (Plot twist: I end up doing this anyway and it works)
Whenever I use the
mysql command, I still encounter the
mysql socket lock error. Therefore, I cannot backup the database.
Here's a list of things I try (using a brute force approach):
- I restart the system; no progress (yes this is an Ubuntu server I know)
- I try to kill the mysql process using
sudo killall -9 mysqland then manually update mysql - still stuck with the lock error
- I update the system with the commands
sudo apt-get updateand
sudo apt-get upgrade- still stuck with the lock and upgrade in progress error
At this point, I don't dare try to directly update Ghost but rather take my time to understand and fix the root cause.
Quick fix solution
Meanwhile, I experiment with an alternative to showing the 502 error. I use Cloudflare to redirect my domain to a landing page I create on Vercel.
I use the following steps:
- I connect my Github account to Vercel
- Using React with ViteJS, I code a new landing page
- I copy and paste links for my social media accounts including LinkedIn and Medium, which I cross-post my posts on
- I deploy the landing page on Vercel for free (you can alternatively use Netlify or GitHub pages)
- I add a new page Redirect rule on Cloudflare to redirect my domain to the new landing page
What I learned
During this process, I learned two things in particular:
- I have no idea how and why Ghost and MySQL decided to update automatically
- People are bad at estimation
What I've noticed throughout my career as a professional Software Developer, is that people are generally bad at estimation - including myself. I thought this update would take an incredible amount of time and therefore kept procrastinating.
Working on the landing page and setting up the redirect rule took me less than 20 minutes, which I did during a relaxing tea break.
In hindsight, if I attempted the first part of the solution as described above, this would have taken me less than 5 minutes. This is something JoKi mentionned in one of my LinkedIn posts.
I should have listened to him and addressed the real root cause.
What I will do next
Now my next move is to update my Blog from the current version, which I think is Ghost 4 to the latest one Ghost 5 (at the time of this writing).
My safe bet is to create a new droplet on Digital Ocean and then migrate my current blog to the new one.
Another possibility is to back up everything on my current droplet and then update Ghost and most probably the NodeJS version. Once done, I can then restore the backup.
I'm also tempted to just duplicate the droplet and then try updating Ghost and NodeJS on the new droplet. If everything works fine, I can then delete the old droplet and in Cloudflare, I can then redirect my domain to the new droplet. Otherwise, I just delete the new one and restart it again.