How to scan a server faster on shared hosting

There are three ways to make a Disk scan in FTP by Zaweba quicker on shared hosting. Set the site's scan connections sensibly, with 3 being the safe number on shared hosts. If the site uses SFTP with shell access, the scan can list the whole tree with one find command. And there is an optional PHP scan that lists everything in a single request over HTTPS.

A plain FTP scan has to ask the server for one folder at a time. On a site with thousands of folders, such as one with a large media library or a big dependency folder, that is thousands of round trips, and the time adds up.

Scan connections

Scan connections is a per-site setting in Site Manager. Disk scans run in parallel over this many connections, each reading a different folder. More connections means more folders read at once, up to the point where the host says no.

Shared hosts limit how many connections one account may open. When a scan hits that limit the server answers with a 421 error, and FTP by Zaweba backs off and carries on with fewer connections rather than failing. So a too-high setting won't break anything, but it can waste time on refused connections. Starting at 3 is the safe choice. If your host is generous and scans are slow, try a little higher and see whether it helps.

Remember that your other work shares the limit. Uploads and downloads have their own setting under File > Options > Transfers, and both draw on the same allowance at the host.

The find scan over SFTP

If the site connects over SFTP and your hosting account has shell access, Disk can run a single find command on the server that lists the whole tree at once. That replaces the folder by folder crawl with one answer, which on a big site is dramatically quicker. If the command isn't available, the scan falls back to the normal crawl automatically, so there is nothing to lose by trying.

Whether you have shell access depends on your host and your plan. Many hosts offer it, sometimes as an option you switch on in the control panel. Setting the site to use SFTP in Site Manager is worth doing anyway, because it is encrypted.

The optional PHP scan

On hosts with only FTP, the PHP scan is the quickest route. Disk uploads a read-only listing script with a random name, calls it once over HTTPS with a one-time token, and deletes it straight away. The script lists the files and sends the answer back in one go.

It needs the site's public address, set in Site Manager, so that Disk can reach the script over the web. Because it writes a file to the server for a moment, it is optional rather than the default; if your site's rules forbid any unexpected files, leave it off.

Making long scans less painful

Even a slow scan can be managed. Pause and Resume let you stop a scan while you do something else on the connection and then carry on, and Stop ends it. You can start reading the treemap and the biggest 100 files as soon as the scan finishes, and the list can show just one branch if you only care about a single folder.

Exclusion patterns in Site Manager help in a broader way. They keep folders such as node_modules out of transfers, which makes everyday syncing quicker.

Which to choose

If you have SFTP with shell access, use it and let the find scan do the work. If you have only FTP and the site is big, the PHP scan is worth turning on. For a small brochure site, the normal crawl with 3 connections is quick enough and needs nothing changed at all. Whichever you use in FTP by Zaweba, the results are the same treemap, list and lens.