Before my visit to Russia I was working primarily on the Speeple control panel and blogging services, but because of my limited internet access I worked on a new version of the news bot for Speeple News.

The new and vastly improved version is multithreaded and makes use of the ROME RSS/Atom syndication and publishing tools library rather than my own RSS/Atom parser. I can't be entirely sure of the performance difference; the main reason for using the ROME library was to get development rolling along quickly.

The new bot performs amazingly well in comparison to the old variant. It manages to crawl 50K news feeds within the hour on a server with 8 CPU cores (using 4 threads per core).

There are numerous bugs at the moment that need ironed out until I permanently use “Speeple Newsbot 2.0” in production, but they're minor enough that this should only days.

Key Open Source Java libraries used:

• Jakarta Commons HttpClient
• DBCP Component
• ROME: RSS/Atom syndication and publishing tools