Rupert's DiarySlightly sanitised for the squeamish

Over in San Francisco yesterday, the brand-new NoSQL movement held its first public meeting. 150 bitwranglers from outfits large and small absorbed ten presentations about how to handle data in the new online world. (Disclosure: the meeting was hosted by CBSi, ZDNet UK's own mothership, and organised by Johan Oskarsson from our sister company last.fm.)

The NoSQL movement – if it's ready to be promoted from notion – is one of those regular reactions to the status quo that make life worth living. Very smart people with big ideas are keen to find ways to store and manipulate data that don't rely on the old relational database ideas, or challenge those ideas in ways that aren't appropriate to the standard methods.

There are many overt motivations behind this. Lots of important database applications have arrived with the Web revolution, and many of those have particular size, scaling, performance and reliability needs that might have optimal solutions outside the mainstream. To take the most extreme case – could Google have prospered, or even worked, had it been designed to run on top of Oracle? But that's just Google, right? The NASA of online commerce? You don't get much insight by noting that you can't get to the Moon on a Boeing 737: it doesn't make much difference to the plane you catch to Alicante.

The difference here is that everyone wants to go to the Moon. To be more precise, they want to be NASA – able to pull off the biggest projects at the same time as doing fantastic things with cheap little robots sent all over the place. If your big idea doesn't work on a tiny phone as well as scaling to the point where everyone on the planet can use it, then you're missing out. And "work" isn't just a question of technicalities, it's as much about economic engineering.

That's particularly interesting, because the mountain of maximum economic pain is nowhere near the canyon of maximum technical difficulty – and it's the technical stuff that tends to attract the attention of the young punks of NoSQL. If you're a very small outfit with big ideas, you evolve something very clever because you've got no choice. If you're a big enterprise, you can use your chequebook to beat your suppliers into compliance. It's the guys in the middle who get shafted. You've got mainstream needs but are too small to make waves: the suppliers are much bigger than you, so you take what they give.

Which is why SME problems with database tech – and technology in general – are centred around getting a fair deal. About not being forced to pay too much on very bad terms. About not ceding control over the company DNA to people whose business objectives are not your business objectives. It's a classic balance of power problem, one illustrated by comparing the balance sheets and profit margins of those very large software companies who supply corporate IT with those of the SMEs who consume it.

For NoSQL to properly change the world, it has to be fit to climb that particular mountain of pain. Like the open source movement from which it comes, it will find this the very hardest task. This particular Everest is frustratingly immune to the weaponry of innovation, lateral thinking, energy and idealism unless harnessed to a focussed, patient, strategic assault. And focus, patience and strategy are not attributes commonly found among bright young punks.

Movements do throw up monsters, though, and its the monsters who devastate. So enjoy NoSQL for the thinking, the arguments, the technology, but watch it closely for its very own Gates, Jobs or Ellison. Sometimes, only a monster will do.

While looking around in the strange and interesting world of Microsoft and the NHS, I found this; a page offering NHS employees in England and Scotland (sorry, Wales and Norn) downloadable MS Office for £9. Which is near as damnit free, even in these straitened times.

But hold on. Isn't the NHS the largest employer in the world, after the People's Liberation Army, the Indian railways and the H1N1 virus? It is. Some 1.5 million UK workers are held tight to its matronly bosom. Which means that roughly five percent of the UK's working stiffs are eligable for Microsoft productivity nearlyfreeware - through that one scheme alone.

That's an awful lot, even before you consider how many other companies may be doing something similar. And I thought MS had a principled objection to making software available at low prices? Doesn't paying less cost you more, in the MS world? And what does this do to the rest of the market - can a company with such an overwhelming presence in office productivity software push discount product like that, under the NHS logo? (Yes, clearly it can. But should it?).

The more I look at large software companies and their licensing schemes, the more curious I become about what would happen if a lot more transparency was imposed. It's not a simple thing to do - if you ever feel the need to talk to yourself, try asking Microsoft or Oracle or SAP or IBM about SME licensing rates. And if that's not silent enough, try asking Microsoft about who owns the IP in the stuff it's doing for the NHS.

And finally, for bonus points, take a look at this, one of the most bizarrely dysfunctional flow charts I've seen in a long time. See if you can find five things wrong with it - and then, for a gold star, work out what it's actually telling us about the relationship between the NHS, your money, and Microsoft.

More on this later. Oh yes.

You may have noticed that we've started to get much more involved with Twitter. That's partially my fault - I'm a certified addict - and partially recognition by everyone involved that Twitter has become a very important part of the online environment.

Twitter has turned into one of the most intriguing experiments in social networking of the new millennium. It can simultaneously engage the world in an authoritarian conflict, and let me explain to a receptionist why her telephone wasn't working (more on that later).

What it can't do, seemingly, is make any money.

That's not a unique problem. But Twitter is a particularly fine example: it provides a unique service which is entirely inimical to advertising. It's very personal and designed to fit into the small change of the day, moments you can spend on yourself and are thus extremely precious. Intrusions break it.

Yet it wouldn't work if it wasn't free to its users. And freemium doesn't work – if there are extras you fancy in Twitter, you get them with the many software packages created entirely independently of the service. Moreover, it takes time to get Twitter – until you start to build your personal network, it seems pointless and confusing, random strangers saying silly things. Who's going to pay for that? Moreover, Twitter's biggest splash to date – the Iranian coverage – would not have happened if people hadn't been able to connect for free.

All the standard models of raising revenue fail.

The key to making money has to reflect the key value of Twitter – which is other people. And they can't be expected to pay for the privilege of being valuable to you. On the other hand, you may be prepared to pay for them. People like giving, when it's to those you care about. But for something as seemingly transient as Twitter it can't be very much money – ideally, you'd be dealing in pennies, not pounds. As we know, though, online micropayment schemes don't work. The overhead in time and expense soon overwhelms the revenue. Micropayment only works in the real world of coins and loaves of bread. The social world.

So here's an idea: social microcredits. For people who really get Twitter, the idea of spending, say, six pounds for a year's subscription is not excessive. Fifty pence a month – or a dollar – is an easy decision for anything you use and appreciate every day.

In return for that fifty pence a month, you get a Twitter account that works exactly as it does now – the one for which you currently pay nothing. The only difference is, you also get five other Twitter accounts you can dispose of as you wish, good for a year but open to you to reallocate at a month's notice.

You could sell them to your pals for ten pence a month – or a pint a year – and you'd break even. You could give them to friends and family, so they can get stuck into Twitter and see whether they like it or not. You could give them back to Twitter, as part of a pool of free credits for it to use in promotion, or Iranian bloggers, or whatever scheme it has – if you approve of it. Or you could give them to any other organisation with a good enough shtick.

Why bother? There are three reasons. The first is that people like to pay for things they find valuable. It assures supply, it increases your own sense of worth, it is a signal of your priorities and your involvement. Here, you're not just paying for the service but the community, in a way that raises your status within that community. The second is that you have control of your account. You've paid. It's yours. The third is that you start to have an investment in how the community grows; you've acquired a small but useful say in how things are.

Anyone who's on a free account can, of course, sign up for the six pound deal: you get the account back and they get five donor accounts of their own. The service grows, and Twitter gets more money – as well as a core of involved users.

In effect, a community barter economy is created, managing credits without centralised accounting. For as long as there's a six to one ratio of free users versus paid-fors, Twitter effectively remains free to join and sample, and for the devoted freeloader who doesn't mind taking the chance of not getting another free credit every year there's no need to pay, ever.

There are lots of unknowns, of course, and many variables and variations on the theme. A 'Pro' badge for paid-for accounts. Other bundles. Different revocation models. But the basic idea has the best chance of any of matching what Twitter gives people with what they are prepared to give back, without breaking the drivers that have made it such a fascinating and, increasingly, important innovation. And in the end, any social network will have to figure the social aspects of its service into its revenue model – that's what it's selling.

(Oh, and the receptionist with the broken phone? I was visiting the BBC, and asked the receptionist to call the studio and let them know I'd arrived. She tried, but the phone wasn't working – and there was no way for her to find out why. As I sat in reception filling in time, I read a tweet from the BBC's technology correspondent, in the same building at the same time, saying that the new VoIP phone system had gone down and was being fixed. I showed this to the receptionist, and we made other arrangements. "Is that the same Twitter that's in Iran?" she asked, astonished – and another fan was made.)

With the release of Apple iPhone OS 3.0, a widespread affliction has become a true, World Health Organisation recognised pandemic. I speak, of course, of the almost eroticised fetishisation that many iPhone owners have for their hardware: it is not too far away from paraphilia when they stroke, coo, gaze lovingly upon and even talk to their devices (and no, not during voice telephony).

Many of my nearest and dearest have fallen foul of this unpleasant affliction. But to get it accepted as a proper condition - and thus start the long, difficult process of finding and applying a cure - it needs a name.

But what to name it? Where could I find a suitable concentration of deeply disturbed yet intelligent people, who would understand what I was saying only too well?

Twitter, of course (where I'm @rupertg) . So I asked for suggestions...

You're a creative lot! Here's the current crop:

* iPhreaks @shadowfax1804
* Massive Losers @katiesol
* Begins with W, ends with kers - @FrasSmith
* Autoerotic technophilia @fgalv
* iPhurries @nickhide
* iSpods @liquidindian
* iPhonlies @Vibrosies
* Multi-touched @mikejaydavies
* iParaphilia @sebFlyte
* Maybe it starts with "lo" and ends with "sers". iPhonitism is quite strange, even for a fetish... @jackschofield
* The word for such users is 'Hello' @WGallagher

Me, I go for malaphilia, from mala, which is latin for apple, also malus which is latin for evil (although @RobBuckley says that it should be milophilia to be true to the Greek, you don't get that nice Apple=Evil effect).

Any more?

So Apple launched a new iPhone, with a faster processor, better camera, video, compass, et cetera. The old 3G iPhone is clearing out of stock at $99. That’s what Moore’s Law does for you. It’s not as if anyone’s come close to the consumer experience of the last couple of iPhones – Android is great, but still a gawky adolescent – yet Apple’s always been comfortable competing with itself.

Everyone will be talking about the iPhone 3GS. So let’s have a look at what else is going on – and why it might matter a bit more than that video camera.

Snow Leopard, the long-awaited upgrade to Leopard, is finally on the runway. It’s got lots of new features, including what looks on the surface like decent Exchange support, 64 bit support, Grand Central multicore magic, many sweet interface tweaks. It looks at least as different to Leopard as Windows 7 does to Vista.

But Apple will be charging $29 – around twenty quid – for Snow Leopard, which plays remarkably well against a competitor which, let’s say, wants to compete on price and has just spent millions on television adverts saying so, but habitually charges up to $320 - £200- for a new OS. And has to, because it doesn’t have any hardware sales.

Be nice if Apple cut the price of its hardware too. Oh, it’s done that too - with margin left for later. If there’s anyone who hasn’t yet seen Apple as a real threat to Microsoft, it’s probably time to return to your planet.

Yet even that’s not the most important part of Apple's latest news. Deep in the traditionally dull third party partner part of the WWDC keynote, the real story lurks. There were medical and ebook applications wheeled out, and Apple is “very excited” about education and health. (Not that there’s a recession on.)

Traditionally, health and education are gristly meat compared to the tasty burger of consumer bling. But these two aspects demonstrate both the breadth of the iPhone’s market and its ability to sink into niches - and they make huge sense with the new hardware accessory support and excellent new developer aspects of iPhone OS 3.0. Those make the iPhone a very tempting remote control/viewing device for all sorts of industrial, quasi-industrial and consumer gadgets (fancy an iPhone interface for your set-top box? That works anywhere in the world?). And the iPhone is already a very acceptable e-book reader, which makes it the perfect platform to attract further development. To say nothing of the 50,000 applications already available - or billion plus downloads already delivered by the App Store.

The iPhone, in other words, has become the default ultra-portable application machine. It's become the PC of the pocket. It’s got there because of the app store, because of the SDK, and because it was cool enough to sell in large numbers and so create the market.

You may have noticed what the PC became in the twenty years after its launch.

There are two small differences. Unlike the IBM PC, where the clones made the market take off, nobody but Apple can make the iPhone. And nobody but Apple can put applications on the thing. Those two limitations would have sunk the original PC, had IBM been able to pull them off.

But then the iPhone is around twenty times cheaper than the PC, is very capable, doesn’t have the same need for expansion and enhancement, and Apple isn't being a complete donkey about app distribution.

So Apple will get all the advantages of a universal market and none of the displeasures of losing its income stream or control over the hardware. Given the difficulties, technical and legal, of producing an iClone, Apple’s monopoly looks assured.

And we know what happens when that happens.

Charles and I have donned the lab coats one mo' time for series six - SIX! - of Dialogue Box, and started out by stepping into space. Since a very young Rupert watched Neil Armstrong hop off the ladder, the technology and potential of space has fascinated me - and these days, IT is bound up with orbital fun much more than you might think. It can also go wrong in various spectacular and not necessarily harmless ways, so watch episode one of the new series when it goes up on Monday.

We've also shot episode two, which carries on that theme to look at some rather more unusual space spin-offs that are either in IT or coming our way. It also includes the most esoteric use of an inferred kebab we think has ever been committed to camera.

It's good to be back behind the podium!

It was a sunny weekend, the first for ages, so what's a geek to do? Stay in and upgrade his Ubuntu, of course. Jaunty Jackalope - Ubuntu 9.04 - welcome to my life.

And the hot news? There is no hot news. I typed in the single command needed to start the process for the pre-launch release candidate, and went off to make the tea. The upgrade downloaded the new files, disabled networking, rearranged the furniture, asked a couple of questions about modified files it wouldn't explain and I didn't understand (so I said yes), asked for a restart, and there it was. Half an hour sipping Earl Grey installed six months of work.

The network disconnection isn't something I remember happening before, when you could carry on browsing even as the world changed beneath your feet. But it makes a great deal of sense: just try and characterise the security threat profile when you're upgrading your network stack while connected. Like a crab shedding its shell, you're better off hiding under a rock until the carapace hardens again.

The new Ubuntu? Looks identical, works identically, is practically identical. The only immediate difference visible is the new notification system in the new Gnome, where black boxes pop up in the top right hand corner to tell you, er, things. So far, that seems to be used by email and IM clients - but not Tweetdeck (I presume because Air doesn't know about it), which throws its own black boxes with its own tales of things up into the same top right hand corner, there to do battle with Ubuntu's own. Of the faster boot time, I cannot say - I haven't timed it, and it was plenty fast already.

New filing system? I wasn't that bored. I will try that - and the various beneath-the-hood changes - on a clean machine later. This was an upgrade to a working system, and the fiddling about can come later.

But the most important news is that there was no news. My standard Ubuntu system at home is a beginning-to-show-its-age Samsung laptop, with a once-was-whizzy ATI graphics chip and a few mild quirks around the audio. The upgrade from 7.10 to 8.04 broke video and audio rather badly; 8.04 to 8.10 rather less so, but still with plenty of fun and games (mostly orbiting around ATI).

This time, nothing. Even VMware, which in the past has collapsed into a twitching heap of tentacles on kernal upgrades - moreover, the sort of twitching heap that is deuced difficult to remove before you can apply the fixes - seemed unaffected by the upgrade. My XP virtual machine started up, if anything, smoother than before: I'm used to the start-up music stuttering if I background the VM during boot; this time, it played perfectly.

I upgraded at 6:30. I turned off at midnight (another wild evening), having forgotten I'd done it. Fantastic.

There is an argument - for want of a better word - among creationists, that while microevolution is possible (small features may change between generations of living beings), macroevolution (ie, a dog to a horse) is impossible. The biologist's world-weary reply is, well, a dog never becomes a horse. Things do change gradually - and if they diverge enough, then one day you'll have a dog and a horse from a common ancestor. In short, there is no 'macroevolution'.

Looking at the experience of changing little and often compared to hugely but ten times as infrequently, I must say that the Ubuntu way is more natural, more evolutionary and certainly less disruptive. Whether that is a good pointer towards the survival of the fittest - we'll just have to wait and see.

It's worked before.

Some more happenings in the xG/xMax world of super-duper low power high speed low cost wireless data, and - for once - I can write about them, as they're not anonymous scandal blog postings saying things I can't check in languages I don't speak about people I don't know.

No, this latest intelligence comes from xG itself, which has redesigned its website, produced end of year results and announced that it has FCC approval for its BTS250 base station.

The new web site you can see for yourself. as you can the FCC approval. Interestingly, this contains a smidgeon of information about the actual operational parameters of the base station - which don't seem to match my understanding of xG's patented secret sauce techniques. And the annual report announcement claims that the company's sold 327 base stations, that it's finished testing them with its handset, and that the South Florida deployment continues.

Ah yes, the handset. The TX60 - apparently finished, although not apparently approved by the FCC - appears on the website in computer generated form only. One tiny snippet in the spec sheet is that talk time on xMax is exactly the same as talk time on Wi-Fi, which doesn't really correspond to the promise made about the patented secret sauce techniques that they would give much longer battery life than any other system.

But then, there is nothing tangible or testable in any of xG's updated claims. We wait, as always, for a germ of independent reality before counting the blossoms of the promised land. There is a new PR man mentioned, so perhaps I'll have another go with this one...

It's true that you can get almost anything online these days, even jobs. And there's nothing like the Internet for matching people with, shall we say, particular requirements with people who have, shall we say, particular skills.

Even so, I was surprised to see this job offer appear on a UK site. It is reference CS55581/J50030A00059383 (in case the listing goes away but you still fancy applying) for a "UAV Pilot" - someone who sits in a shed and flies a large unmanned plane at a safe distance from the consequences. There's not much call for this outside certain speciality roles - and as what's on offer is particularly suited for those who have experience working "within military infrastructures", who can take a "Flexible approach and proactive management of problems" and are prepared for "Overseas rotational working", it's not going to be cropspraying. Oh, and "Successful applicants will be required to obtain UK MOD security clearance prior to appointment" - that might also be a clue. But there is a share option scheme, a good pension, and 45K.

Some further discreet enquiries reveal that yes, this is a job which involves annoying the hell out of people far to the east of Dagenham - two months on, one month off - on behalf of a certain UK defence contractor where you queue going in and queue coming out. And as the qualifications required don't actually ask for any formal flight certification but do leave the door open for people with experience of flying 'large model aircraft', it's quite possible that you could get the job even if the closest you've come to actual aerial operations in a warzone is watching Top Gun in Peckham while fondling your copy of Aero Modeller World.

What I don't know, and what my discreet enquiry service is being far too discreet to tell me, is whether this job is purely loitering around the skies being part of the Plausible Deniability Caper -- as in when the CIA ran U2 spy flights across Soviet Russia, so the military had nothing to do with it -- or whether it may involve getting tangled up in some of the more esoteric interpretations of the rules of engagement for armed UAVs. Oh, you know. Killing people when you shouldn't, but without actually breaking the rules.

I'm sure that's not what's going on here. And besides, the job's probably recession proof. Fancy it?

What a great day for a calendar bug! Testers of Jaunty Jackalope, the 9.04 'netbook friendly' version of Ubuntu, have found a problem that puts all users of the popular open source operating systems back to medieval times. And it's due to Ubuntu's very worthy attempts to adapt to all users. Famously inclusive, the new cut of the OS includes a modification designed to support netbook-toting Berber farmers, the last secular users of the Julian calendar and as such uncomfortable with the Gregorian calendar reforms introduced in 1582.

It'd be easy for us sophisticated Gregorians to mock, but it gives me the perfect excuse to launch into a whole load of religious mathematics and geopolitics – and I'm not one to pass that up. While the Julian calendar had been in use for more than 1600 years previously – ever since Julius Caesar adopted it to match the state of the art known to the early Roman empire – it didn't match the actual orbital period of the earth around the sun. By 1582, the months were around ten days adrift of the season and thus increasingly annoying to a world coming to terms with a far more intricate model of the cosmos. Pope Gregory XIII, in conjunction with his priestly astronomer Clavius (a name familiar to students of the Moon), was annoyed by the problems this caused with the calculations for Easter Day, and so pushed through his famous reforms. This led to eleven days being dropped from the calendar across the Catholic world: England, being a bit sensitive about such things in the late 16th century, held off until the middle of the 18th before adopting the reform – "Give us back our 11 days!" protestors demanded, even so – while the last state to hold out, the very Orthodox Greeks, finally caved in 1923.

But those Berber farmers still stick to their Julian guns, with their fellahi calendar. And open source is nothing if adaptable to suit its users – which, in these days of cheap netbooks, most certainly includes nomadic African agriculturalists. So it made perfect sense to include extra code to make Ubuntu compatible with Julius Caesar's diary. Take that, Outlook!

Now, as anyone who's had to codify time will testify, there are a million gotchas in coping with the many oddities that happen when humans try to synchronise with reality. Those who flit between the UK and the US on business will know how many ways computers can mangle transcontinental appointments, and that's a fairly simple business. To track an entirely different calendar is a major undertaking – one that the Ubuntu coders rose to magnificently. The new code was tested thoroughly and found to work very well: the side effect, that it works with some remaining Orthodox liturgical timetables without obscure plug-ins, was also appreciated by those to whom it matters.

It's just a shame that the relatively simple business of switching between calendars, being trivial and thus a bit boring, wasn't tested nearly as well. A previously unimportant quirk of the leap year detection in Ubuntu, which is supposed to just check the year before deciding whether there's a 29th of February but instead does some baroque recalculation based on date variables and the number of seconds since the birth of Unix, gets it wrong with the new system. It's not the first to have problems with Unix's timekeeping, but this one plunges Gregorian Jackalope users into the Julian calendar three days after what would have been the 29th of February on non-leap years.

That's today. So if you're running Jaunty Jackalope right now and haven't looked, then – aha! -- you've got the eleven days back that the peasants had stolen from them. You are also perfectly equipped to farm among the Berbers. There is a fix, of course, but I think it's only fair to give the chap most to blame for this whole business the last word.