Today’s big buzzword is “Big Data”. Its companion buzzword “Analytics” is also a big favourite.
There are good reasons why this is so. The last five years have seen a massive upsurge of data; we’ve generated 90% of the entire world’s data in the last two years.
This change has been dramatic. Twenty years ago, we had all the data analysis tools in place, but data was scarce. Today there is a frightening abundance of data, but nobody quite knows what to do with it.
What do we do with all this data? What can we do with all this data?
Think, for example, of telecom data. A near infinity of data is being generated, but, till recently, only a small fraction was actually used – and in most cases for billing. The rest of the data simply flowed away, because it wasn’t profitable to either store or exploit the data. If it cost you hundred dollars to store and analyze data, and you could at best earn fifty dollars from it, why would anyone take the plunge?
But now things are changing. Data storage is getting dramatically cheaper, and data retrieval is getting unbelievably faster. So there’s now a good chance that your 100 dollar investment will return you 120 dollars instead.
That’s why everyone is jumping onto the bandwagon, and everyone’s making wild promises. “We’ll make your data talk and sing”, one analytics enthusiast recently told me.
I’m predicting that there will be no easy song and dance. Analytics isn’t a magic wand. There’s a lot of hard work ahead before the party can start.
Let us start with the nature of the data. You can’t ‘read’ data as you might read the next book from Jeffrey Archer. Telecom data is an endless stream of zeros, ones and mumbo-jumbo. Hidden among this mad rubble is a great story, but you need immense skill to ferret it out of the impossibly hard looking data pile.
The data must first of all be ‘collected’, i.e. pile together all the useful pieces from the rubble, and discard everything that’s unnecessary or irrelevant. Next, it must be ‘mediated’; that is we must transform the nonsensical looking data into something understandable and usable. For example, be able to say that the subscriber with number 123456, spent so much on local calls, so much on long distance calls, so much on text, so much on data download and so much on Flappy Bird before it was withdrawn.
So who is this subscriber 123456 anyway? Where does he stay? How old is he? Is he even a ‘he’? Clearly a telecom switchboard isn’t going to give you that information … but there must surely be a database somewhere that contains the coordinates and personal attributes of this unknown person. We therefore ‘enrich’ 123456’s telecom data by joining it to his personal database.
Things may get even harder. Suppose there was a data ‘blackout’ for some 15-minute spell during which some interesting event occurred. How can we deal with such missing data? Can we model and hypothesize what this data could be? Could we estimate the probability that our estimated data is right?
Collection, mediation, enrichment, costing, modeling, pricing … all these are challenges that the telecom analyst must grapple with. All too often we are led to think that analytics simply involves pouring data into a calculating furnace and coming out with exciting and lustrous inferences that will save millions of dollars! This is simply not true.
While the drama, romance and mystery seems to be about models, algorithms, visualizations, animations and inferences, that’s merely the glossy end of the picture. Real success is assured only if the data preparation, aggregation, mediation and enrichment happen correctly. This backroom pain and labor is actually responsible, I believe, for almost 90% of the eventual success.
In the final count, telecom analytics is about telecom awareness, telecom domain knowledge and hard-earned telecom experience. It is only after you create this painstaking edifice that the song and dance can start.
The best telecom software companies have successfully created this riveting platform for song and dance. We’ll end this note by looking at some virtuoso performances on the telecom analytics stage.
Somewhere in the middle of nowhere in America
Every US citizen has the right to connectivity – even if he stays in the middle of nowhere.
For telecom carriers providing this statutory long-distance connectivity for the ‘last mile’ can be quite a nuisance. So long distance telecom carriers usually prefer to sign up with local carriers to assure this last mile connectivity.
The local carriers charge a very stiff rate for this last mile connectivity (e.g., 5 cents per minute, instead of the normal 0.5 cents per minute), but that is justified because traffic is expected to be very low and they need to be profitable.
But sometimes the local carrier can get greedy! Wouldn’t it be great if he can somehow contrive to ‘pump up’ big traffic along this normally deserted last mile? What if he tied up with someone offering an adult chat or conferencing service? That would send traffic soaring!
In a celebrated legal battle some years ago, analytics successfully came to the rescue of a US long-distance carrier suffering losses adding up to many millions of dollars. Of course it required considerable telecom acumen to prepare the evidence needed to satisfy the court of law.
From the switch to the bill
How do telecom carriers determine what rate to charge their customers? In most cases – and this is a bit of a surprise – this rate is based on what the competitor charges. But is this rate right? What is the telecom carrier’s true cost?
Answers to such questions aren’t easy because they involve a detailed breakdown of all the individual costs: the cost of connection, termination, circuits etc., etc.; and making such breakdowns again requires the intelligent use of the different elements of telecom analytics like mediation, costing, enrichment, and billing.
Often such analysis results in unexpected findings: for example that the big revenue earning trunk route is actually a money loser!
The anatomy of a telecom traffic jam
This example may still belong to the realm of fantasy, but it is something we are sure to see in the future.
Telecom carriers provide all their services via a very complex network of devices. By positioning sensors at various locations in the network it is possible to ‘visualize’ exactly how the network looks like (the comparison is tenuous, but think of a helicopter hovering over a city and taking pictures of traffic movement).
Some more sophisticated analytics will help us classify telecom traffic patterns … and for every worrisome pattern we could provide a network ‘re-routing’ antidote to ease the congestion. Indeed it could even be possible to configure a ‘learning’ network that is continually readapting itself so that it always stays efficient and error-free.
These examples are pointers to the tremendous possibilities that big data telecom analytics offer us now, and in the future. But we must never forget the underlying secret: only blue-blooded telecom experts can usher in this exciting telecom future, just as only an accomplished doctor can perform the wondrous life-saving surgery.