Saturday, November 22, 2008

Performance of copying files to an external drive

Did some tests to see if the claim of the "fast copying" programs are true. Below is the results I got from copying a Firefox profile from a laptop hard drive (NTFS) to an external USB2 hard drive (WD Mybook Home 1TB as FAT32) on Windows XP SP3. The Firefox profile was 18.6MB (19,552,413 bytes), 630 files & 174 folders. This contains lots of small files, which usually hurt performance when copying & deleting. I copied 5x100MB files from the laptop to the external drive to make sure there is nothing in the drive buffer (which only takes about 10 seconds), to make the results fair. The WD Mybook Home has Firewire400 & eSATA ports as well, but I didn't try those (but I have done some informal testing in the past, and USB2 & Firewire seemed to be pretty equal.)

The Results:

ProgramversionTimeMaintain folder dateFree?Notes
Beyond Comparev2.4.3209.8sYesNo
FastCopyv1.9196.1sYesYesdefault 32MB buffer
TeraCopyv1.22426sNoYesdefault (to another HD)256KB buffer
WinXP explorer copy97s (stopwatch)NoYesExt. HD still spinned for a while afterwards
Based on the results, FastCopy is my new "copy bulk amount of files" program of choice.
Please suggest any other programs or configurations to try...

Monday, October 13, 2008

First update on mapping OpenStreetMap (long overdue)

Australia » Victoria » Melbourne
I signed up for OpenStreetMap about a year ago, and there were plenty of warnings that it's very addictive, but little did I know!!!
I do mapping in spits & spurts (like a lot of other people I'd imagine).


Turns out I have done a fair bit since I signed up:
Melbourne contributors
Looks like I'm the biggest contributor in greater Melbourne, scary stuff.

I've traced streets, added a lot of names and added some POI's like parks, foot/bike paths, pubs, schools etc.

Rough order of Suburbs I've done:

  • Essendon
  • Moonee Ponds
  • Aberfeldie
  • Strathmore
  • Ascot Vale
  • Flemington
  • Travancore
  • Kensington
  • West Melbourne (industrial area)
  • Strathmore Heights
  • Gowanbrae
  • Airport West
  • Keilor East
  • Avondale Heights
  • Maribyrnong
  • Maidstone
  • West Footscray
  • Footscray
  • Braybrook
  • North Sunshine
  • Sunshine
  • Keilor Park
  • Tullamarine

Since I don't have a GPS, I trace from Yahoo! imagery and fill in as many details as possible.

My usual steps:

  1. Trace streets, parks, foot/bike paths from Yahoo! imagery
    • Add as many tags as is obvious, leave alone if in doubt
      • eg. highway=residential, source=yahoo
  2. Ride along "main" roads to pick up all the side street names, scribbling on my Palm T3 Notepad application (exports to PNG)
    • Tag as name=blah, source:name=survey
    • Later on use Maplint or NoName layers to spot missing street names
      • Plan another ride that swings past all remaining streets
  3. Walk or ride down *every* street, after printing off a Mapnik layer map of the area
    • Note down:
      • All street names/references/lanes/bike lanes/speed limits
      • Schools, park, shopping centre names and shapes (to refine the rough tracing from Yahoo!
    • This takes a long time, and I've only done a little bit so far, but it does make the area almost 100% complete


  1. Either Yahoo! updates their Melbourne satellite photo's (~5 years old) and provide higher zoom levels (currently level 16) or Google provides their photo's to OSM
  2. Potlatch is my editor of choice, and it has improved a lot since I started using it
    • Zoom further in by scaling the Yahoo! tiles when hitting the highest zoom Y! provides
      • Potlatch used to do this, which made it much faster to trace, but a Flash bug that occurs when the auto-maximise feature was added means this had to be turned off :(
    • More tag "autocomplete" entries (especially source:name=survey/photo etc.)
    • When crossing a way with another (either as a T-intersection or cross), I think the snapping feature of Potlatch means the created node is not where the two ways mathematically cross, but the nearest snapped point
      • So the angles of ways change a bit, needing zooming in to fine tune
        • But snapping is very obvious at z18, which becomes dodgy when ways are close to (but not) orthogonal
      • I usually layout a grid of streets with a little overlap, then come through and join all crossing ways, crop T-intersections overhang and fine tune
  3. Get a GPS and:
    • Circumnavigate local parks etc. which are hard to see the boundry from Yahoo! imagery because of trees
    • Enter POI points like postboxes, speedhumps, changes of speedlimits
    • Take photos as I go of all the street signs, amenity names etc. which, with a GPS, can be geotagged (and upgrade name's to name:source=photo)


I've noticed that there is a fair amount of pride/possessiveness around the contribution that OSM users have made, and I'm no different.

I get annoyed when other users come through and add inappropriate tags (not what was on the ground, "for the renderer", not "best-practise" etc.), move carefully smoothed ways around/offset. I guess the "pride" of one's contribution is a double edged sword.

Good tools for letting me know when someone has editing "my areas" would be fantastic, and I've just signed up for Itoworld's osmmapper (RSS feeds of changes to specific areas). But I think it's hard to communicate what has changed, how do you convey changes in a concise/meaningful way?

  • eg. Mary St is now oneway & nodes moved; Frank Rd deleted; Railway Hotel pub added+tagged opening hours; etc.

Crossposted at OSM diary

Friday, July 11, 2008

Palm T3 Documents To Go problem

I sync my Palm T3 on 2 computers, which sometimes cause problems after I've had a hard reset (a complete data wipe). This time it was because it ran out of battery juice (I suspect - a button was held pressed when packed in a bag).

I mostly fixed all the issues (after having to manually hard reset again after not entering the correct time/date - stuffs up the forced sync for some reason), but Documents To Go (v6.008) sometimes gets confused about syncing existing documents.

The error message I see is:

Another file exists with the same name as this one...

But there is no documents with the same name!!!
DataViz support had nothing in their Knowledge Base. I have seen this with both "Sheets to Go" and "Microsoft Excel" formats.

I tried everything to fix this, but was left with the poor option of just changing the name of the document to something unique. But today I found a solution

Using either Filez or Resco Explorer, I deleted all of the Documents To Go Palm databases. I think that DocsToGoDB.DATA is the one that solved the problem, hard to tell. It did cause all the categories and their assignments to disappear, but easy for me to add again. So I deleted:

  • DocsToGoDB.DATA
  • DxtgTempDB.TDAT
  • (in Resco Explorer) all Documents [DTGP,xx].pref "files"

I was then magically able to see the "existing" files of the same name, delete them, rename each document to their "proper" name, assign a category and sync properly!!

Wednesday, June 18, 2008

Firefox 3.0 released, download now!!

The Mozilla corporation/community have released Firefox 3.0 today, with many improvements (both UI wise and backend changes).

Similar to Firefox 2.0 & 1.5, this release has had some serious amount of QA pounding on it, I would say it is their most planned & executed release management to date.

The things that interest me for this release include:

  • Awesome bar (smart location bar)
  • one-click bookmarking
  • Tagging history/bookmarks
  • Better password manager (not getting in your face)
  • New EV certificates (better identification of secure sites)
  • Add-ons Manager with inbuilt Addons listing
  • Download Manager improvements (mainly resumable across sessions, more robust backend)
  • Full Zoom (images & text zoom together)
  • Improved Memory Management (cycle collector, reduced leaks, jemalloc allocator to use less memory)
  • Faster JavaScript
  • Faster Page Load
  • Advanced Graphics (reflow rewrite, better SVG/Canvas support, better graphics backend (Cairo))
  • Future Weave sync'ing

I only started using 3.0 with the release candidates (RC1), because of the "addons not yet updated" problem. But I found updates for my day-today addons during RC1, so upgraded.

Finally Firefox 3.0 has places support, which I think will make a real difference to users web browsing experience (mainly through the "Awesome bar"). Here's my anticipation from earlier:

When Firefox 1.5 came out, I was already excited about 2.0, what with Places & the SQLite backends, but that didn't really pan out, should be in Firefox 3.0 though...

Firefox 1.5 released today
... Already waiting for Firefox 2.0 (from the 1.8.1 Gecko branch) with its "Places" support (redoing bookmarks, history, feeds etc. into a new user interface and storing in a super fast SQLite DB).

Firefox 3.0 will come off the 1.9 branch, with it's massive Gecko improvements (rewrite rendering using Cairo on all platforms, reflow rewritten, complete SVG support??) and the aforementioned Places with tagging system for bookmarks/history etc.

Get Firefox!Download Firefox 3.0 now.
Hopefully download before 3am, 19th June AEST to create a new Guinness world record for the most downloads of a software program in a day!!