Josh Freeman

Web Developer

Computing Science BSc(Hons)

Get my CV!
JoshFreeman.me 2013 © all rights reserved
The advantages of location specific parsing
posted 28th January 2013

One of the major issues I encounted on XIVPads was that I never liked how long it took to parse a single webpage. A single page took 0.9 seconds on average and when updating a characters data, there would often be over 10 pages to parse. After fetching the page, extracting data, comparing, conclusion maths and database inserts, it could create a nasty 20 second or higher wait time for updates.

The reason came down to something simple, location. Fetching pages from Japan using a server in the US is not optimal and so in the latest version of the website a location driven "Sync" system has been developed so that a Japan server can parse Japan web pages and once the data has been formatted and compressed it can be sent to the US server.

The above demo illustrates the time difference, with the current US parsing on the left and the much newer Japan parsing on the right, the time difference is extremely improved when using a large scale "Sync" system.