I upgraded my instance (at scicomm.xyz) to 3.3.0 about a week ago, following the instructions on the release page. I didn’t encounter any problems or errors during the upgrade. However, today, a user on my instance reported that the API isn’t working correctly.
Indeed, visiting https://scicomm.xyz/api/v1/ results in the error-phant image.
However, checking https://scicomm.xyz/api/v1/accounts/1/statuses works.
The front-end is working fine.
Restarting nginx and/or the Mastodon processes hasn’t fixed the issue.
journalctl -u mastodon-web -f I can see:
path=/api/v1 format=html controller=ApplicationController action=raise_not_found status=404 duration=12.06 view=2.30 db=1.63
And also in the nginx access.log:
xxx.xxx.xxx.xxx - - [08/Jan/2021:11:06:42 +0000] "GET /api/v1 HTTP/2.0" 404 647 "-" "Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Firefox/78.0" xxx.xxx.xxx.xxx - - [08/Jan/2021:11:06:42 +0000] "GET /oops.png HTTP/2.0" 304 0 "https://scicomm.xyz/api/v1" "Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Firefox/78.0"
The last major change to my system was the upgrade to v3.3.0.
I am not too sure where to go next in troubleshooting this issue so any advice would be appreciated!
I also seem to have lost some admin functions, e.g, from Settings, if I click Federation I also get the error image, so I cannot perform any domain-block management. It is starting to feel like something went awry during the upgrade process but any comments would be helpful.
https://mastodon.social/api/v1 is not a good endpoint to try (there is no such endpoint).
Any other endpoints that do not work?
How did your database upgrade go?
Ah, sorry, I meant https://scicomm.xyz/api/v1/instance no longer works.
Also https://scicomm.xyz/admin/instances?limited=1 similarly returns an error page since the upgrade.
The database migration did not report any errors.
I have backups going back a month so I am prepared to roll the whole instance back to 3.2.2 if it might be too tricky to fix. This would lose a few posts from the past week before the problem was reported.
Can you peek at the logs in response to those two API requests?
The nginx access.log returns a 500 status in both cases above.
PostgresSQL reports, for /api/v1/instance:
2021-01-10 17:30:11 UTC [19920-1] mastodon@mastodon_production ERROR: materialized view "instances" has not been populated
2021-01-10 17:30:11 UTC [19920-2] mastodon@mastodon_production HINT: Use the REFRESH MATERIALIZED VIEW command.
2021-01-10 17:30:11 UTC [19920-3] mastodon@mastodon_production STATEMENT: SELECT COUNT(*) FROM "instances"
and for admin/instances?limited=1:
2021-01-10 17:32:09 UTC [14534-91] mastodon@mastodon_production ERROR: materialized view "instances" has not been populated
2021-01-10 17:32:09 UTC [14534-92] mastodon@mastodon_production HINT: Use the REFRESH MATERIALIZED VIEW command.
2021-01-10 17:32:09 UTC [14534-93] mastodon@mastodon_production STATEMENT: SELECT 1 AS one FROM "instances" INNER JOIN "domain_blocks" ON "domain_blocks"."domain" = "instances"."domain" LEFT OUTER JOIN "domain_allows" ON "domain_allows"."domain" = "instances"."domain" LIMIT $1 OFFSET $2
I have posted this on Github Fix performance on instances list in admin UI by Gargron · Pull Request #15282 · tootsuite/mastodon · GitHub
Maybe you can try to connect to your PostgreSQL
mastodon_production database with
psql and try
refresh materialized view instances
That has solved both problems
Thanks for your help!!
Great it worked - however I am still curious why this happened in the first place.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.