So here I am on day 1 of my 30 day challenge to design, build and release a new software product within 30 days. I’ve got a few quiet weeks (i.e. no major releases planned for existing products) and already have one customer signed up to use a service similar to one I’m already using for in-house reporting and monitoring. Only our in house service is MySQL/PHP based (so doesn’t support .NET Web Services or WCF) and since it’s only me looking at the stats – the interface is rather raw. Compared with the existing web stats services, it’s ugly.One method I’ve recommended to software developers in the past (and one that I’ve used rather successfully) for a quick and dirty tracking solution is to sign up for one of the many free web analytics services, add their custom JavaScript code to an empty web page and make a HTTP GET request to this web page whenever you want to register a “hit”. By getting this secret page each time your software starts, you can generate a rather pretty usage chart such as this one from StatCounter:
This data immediately tells me where the software was used (by plotting IP addresses against a list of known locations), when it was used, and how many times each day it was used. What it’s not so good at is determining returning visitors. This is because the cookies used to “remember” a visitor get lost between “visits”.
The same technique can be easily applied to Google’s Analytics, or Goingup’s Web Stats.
One of the other products I have designed, the Insite123 tax return software – uses an explorer style interface but on startup loads a web page with latest news and service status on it. A .NET 2.0 web control is used to retrieve this page and display it to the user. The page also contains the JavaScript to log the “hit” on the web analytics package.
Great minds
A friend of mine recently pointed out to me that another of the 30dayer’s is also doing a desktop software analytics package. Of course this is great news; web services for desktop applications have been heralded by all the major software companies as a breakthrough in how software is used and consumed and yet there are very few services available for developers themselves to consume from other developers in order to provide a better all round desktop product for their customers. Since someone else also has an idea to do a similar thing, I wonder if these types of software as a service type products are going to be the way more products will be delivered in the future.
Error Reporting
Anyway, web or software analytics is only half of the story. What a software company needs is detailed and recorded data, especially for error reporting.
What I do at the moment is to trap every single exception and silently attempt to report the error to my PHP web service via a RESTful XML POST or HTTP POST which in turn logs the detail error into a MySQL database. The MySQL database is reported on to give me the three most common errors of the day/week/month. I regularly go through this “event log” and inspect the error messages, the stack trace, the inner error (perhaps a third party component threw the original error), and also any custom application settings the user might have. This way, I can reproduce the error or at least prevent it from occurring.
I use Microsoft Access in order to connect to my remote MySQL database and secure the connection by IP restriction (so only my home office network can connect to the web server).
If however the user gets an unhandled exception, I do display a custom error dialog with the exception text and provide a button for the user to report the error and simultaneously open a support ticket. The error will also be logged via the online service. The great thing about having a back-end server receiving diagnostic information is that when a user calls to complain about a bug it is possible to inspect your error event log database and see the error for yourself – exactly as you would if you were debugging your own code.
Other benefits
The benefits of an online server service for your software doesn’t end here either. I have expanded my own in-house server to incorporate software licensing. For example, Gateway Solver is a product which allows UK businesses to file their tax returns and interface directly with government agencies online. It has a number of modules for each type of tax or agency. Each module is paid for separately on a subscription basis and can be topped up in various quantities. Each piece of software is tied to a specific machine and can file for a specific subset of companies, since sometimes one company in a group might want to do all the administration for all of the companies.
Such a system cannot be handled easily using conventional “license keys”, and I’ve seen quite a few Micro ISV’s catering for these larger corporations having to roll their own mechanism or use a restrictive off-the-shelf service for license key / account authentication.
For example, when running Gateway Solver, a customer simply selects the “My Account” button in order to query the online service in order to determine the state of their subscription. Their subscription effects what they can and cannot do with the software.
Potential Snags
Clearly privacy is an issue here. I don’t like software that has a phone-home component, but unfortunately remote services are becoming a fact of life in software. Almost all modern software makes checks for latest versions (usually by downloading a file from a website and comparing the date/time/version) or to check license key subscription status. Clearly the service needs to only allow a tight subset of data to be logged and if the customer is using the service for license/subscription challenging, more thought needs to be applied by the developer on what to do if the service cannot be reached (for example, if ZoneAlarm is having a bad day). The basic technology of using HTTP POST or GET to send and receive XML data has been used in my software since early 2005 by quite a large group of people and only a couple (those using HTTP proxies) have caused me to have to re-code to avoid the issue. My own software will not function unless the online service can be reached – this is a condition of the license – the subscription is a service and not owned. On the other hand, if some people have very restrictive firewall policies it is possible that a customer of mine using this online service to “log” a “hit” the packets of information might not get through. This is probably not a big deal, since it would only account for a very small minority of “lost” information each time. Although, to be honest, in my experience the mechanisms that I will be using are unlikely to be flagged as Spyware – across a dozen or so products, ran literally a million times or more – they haven’t yet…
What I intend to do is provide a service for other software developers to perform the following:
With 29 days to go I should say now – if you’re interested in a beta test of this service, please let me know! You can contact me in the comments section here, or via email to: mike [dot.] wilson [{at}] evolvedsoftwarestudios.com.
So there I am, procrastinating in Starbucks, enjoying my large frozen caramel and ice-coffee mix with Vicky and Dylan and we suddenly realised that Dylan was fidgeting in his seat a bit too much. At first I thought he’d just sat on one of the starbuck’s special chocolate cakes, but sadly this was not to be for he had performed the largest poo we had ever seen.
A very messy change in the back of the car ensues and we realise that it’s a blessing to at least have fresh air outside when doing a nappy change.
I’m back in the office now, air conditioning on full blast and 1,000 pages of an ASP.NET 3.5 manual to scour and digest. I reckon I’ve got a couple of days to figure out how to implement ASP.NET master pages, profiles and get the initial web farm up and running.
Speaking of farms, Vicky and Dylan are off today enjoying the beautiful Farnham countryside, they’ll probably visit a farm. Dylan smells the part, but I do miss being able to go out with them, especially on a hot sunny day like today.
But I only have 29 days to go.