Since writing about WikiApiary last, I didn’t have much time to work on it. In the meantime, denizens of the MediaWiki Stakeholders chat room (join us!) have mentioned more than once that it wasn’t very responsive.

In the past few days, I finally took the chance to look at in terms of server resources rather than just an SMW or website problem. This way finally led to some results.

Load was pretty high on the machine even though there wasn’t that much traffic. We’re talking a load in the 30s or so. Running top showed that there were a lot of apache processes running, consuming a lot of memory and sending the machine into swap.

A quick investigation into apache settings showed the problem:

       MinSpareThreads          25
       MaxSpareThreads          75
       ThreadsPerChild          25

Changing those settings to 5, 25 and 5, respectively, meant fewer resources were being used out of the gate, so load quickly fell to below 10 and the site became more responsive since there was less thrashing going on inside the box.

We still have work to do on the bots. I’m going to be working with at least two other people on that this year. One of the first steps is using the Cloud Services to set up the crawlers.

Photo credit: Vitalii Bashkatov, CC BY-SA 4.0 https://creativecommons.org/licenses/by-sa/4.0, via Wikimedia Commons