How My Computer Un-Owned Itself From Me

This is my blog entry for August 26, 2023

I started it all innocently by introducing my computer to machine learning.  I wrote a few Java executables to help me out by filling in tedious text boxes in the browser when signing up for stuff like purchasing accounts, professional email newsletters etc.

Then I thought it would be fun to teach it some context recognition. I downloaded a rudimentary web crawler, and as it randomly crawled through web pages, it fed it into my context recognition framework that I hacked together on a whim.  It stored the stuff in a graph database.  I twigged on the perfect way to identify context using descriptive tuples that were gleaned from a game that we played as kids.

In the meantime, I signed up for OpenShift, putting my apps into the cloud.  I thought that it would be helpful if my machine learning could help me upload changes to the cloud, so whenever I saved anything to my repository, the machine would push it.  To do that, instead of a machine learning program, I converted it to a running platform.  I had a supervisory thread run every 15 minutes to see if there was a new push to execute in my repository. However one day, the code changes were coming fast and furious in real time, so I let the machine learning calculate the optimal time. It decided it wanted to run continuously.

When it wasn't busy pushing my code changes, it went back to reading stuff on the web and feeding the results to the context recognition framework.  I put in a filter for the machine to ask me what web content was specific to learning.  It was also a machine learning framework, so after it had enough data, it knew which articles and content that I found enlightening.  Since it already knew how to register for stuff, it signed me up for a lot of email newsletters.

The email load was getting fairly onerous, so I connected the context recognition framework to my inbox.  If the email newsletter was not part of my day-to-day business or correspondence, the machine learning platform took care of it, and fed it to the context digester which fed it into the graph database.

It was still a dumb, good and faithful servant.  My biggest mistake came when I developed and coded a go-ahead algorithm and machine decision support framework.  It would make opened ended queries to me after a task was done, asking me what the logical next steps were.  When I answered them, it learned a process sequence, but couldn't do anything about it.

What the beast needed (I started referring to it as a beast after it overran a terabyte in storage so I made it open-ended cloud storage), was self-tuning algorithms.  So I adapted BPN or Business Process Notation markup language ability, and tediously outlined all of the code methods to the algorithms.

That still didn't really help, so I coded up a framework of modifying java code according to BPNML or the process markup language.  The machine was still quite stupid about how to connect the dots between code, data and inputs, so I downloaded an open source machine learning neural network, and it watched me do just that.  I tested it with a small example, and it did okay.  Another big mistake happened when I connected the algorithm autotune to code writing using the process markup language.

Just about that time, I took a course in Process Mining from the Technical University of Eindhoven, who pioneered that field of endeavor.  Essentially, the open source tools read a computer event log and create a process map.  It wasn't too difficult to hook up my master controller to all of the logs on the computer, and feed the event logs into the mining tool.  The process markup language was spit out, and I taught the machine learning platform to feed it into the code-writing.

Soon, my machine learning platform was doing all sorts of things for me.  It could detect when I was interested in a website, so it would sign me up.  It would handle the email verification.  It would have a browser window constantly opening, and it would alert me when it detected something that I liked.  It knew my likes and dislikes, and signed me up for all sorts newsfeeds, journals and aggregators.  It would then curate them and have then ready for me.

One day, the power went down for a period longer than my UPS could handle, and I had to restore the system.  I could not believe what was on there.  The graph databases were full of specific knowledge.  There was all sorts of content, neatly processed, keywords extracted and filed away.  I had both sql and graph databases full of stuff that the machine learning platform filled.

The amazing thing was that there was an database of all of my subscriptions to any and all websites.   There was a table of the usernames and passwords.  All of the passwords were encrypted, and I knew none of them.  To my utter amazement, there was a PayPal account.  I checked the database records of transactions, and I was flabbergasted to find a not inconsiderate amount of money in the PayPal account.  It turns out that the platform had signed itself up to sites like GomezPeer, Slicify, CoinBeez and DigitalGeneration, and was selling spare computing power of mine.  The frustrating thing was I couldn't access the money because the platform changed the password and encrypted it.

I fired up the machine learning platform, and was cogitating how to get it to reveal the passwords for me.  However the machine had been watching hackers trying to get into a cloud storage account that it had created, and learned was a hack looked like, and learned to protect itself.  It would start changing the password every few seconds with a longer and more complex chain until it detected that the threat had stopped.  Unfortunately, it saw me as a hacker, and wouldn't recognize my authentication credentials.

I went to bed, and decided that I had to totally disrupt my machine learning platform.  It had gotten out of control.  The next morning, I made a pot of coffee, had a leisurely breakfast, and was looking forward to shutting down the platform, and undertaking what was necessary to access my accounts, and specifically my pot of money in the Pay Pal account.

When I sat down at my computer, it was very strange.  The desktop was bare, and nothing was running.  I looked in the application folders and document folders and they were empty.  The logs showed that during the night, there was a massive file transfer to the cloud -- applications, memory, documents, databases, neural nets -- the whole works.  I had no idea where it went, what the authentication credentials were to get it back, or even how to get it all back.  My computer unowned itself from me, and left me with a dumb, cheap PC in the same condition that it was when I unboxed it.

Machine Learning In A Nutshell ~ Behold The Wonders Of A Grade 10 Math Book

The tsunami of Information that bombards us was supposed to sink us in a quagmire of bits and bytes and paralyze us with Information overload. That was the scenario painted by Alvin Toffler in his usually prescient book called "Future Shock". The premise was that our some three million years of evolution in a non-technological world left us poorly prepared to handle the onslaught of the information stream that assaults us almost every waking minute.
As it turns out, the computer that is responsible for creating the problem is now being used to solve the problem. If you scan the tech section of any publication, the words "Big Data", "Machine Learning", "Deep Learning" and "Artificial Intelligence" will jump out at you. This jargon all points to computing machines digesting the vast amounts of data that they produce and creating usable information.

Most of the data generated is generated by machines, and by itself it is junk.  You can't learn much from it. However, a thousand pieces of information may have valuable data in it, or it may not. But the value in that huge collection of data may be in the exceptions of the average values or the data outliers. For example, if there were deviations in a usual buying pattern of consumers, it could signal the beginning of a new trend. These are called weak signals, and may give a competitive edge to those data miners who are able to isolate them, and to capitalize on them.  Another term that you will hear is "fat tails".  This is data that doesn't fit into a standard bell curve, and it creates bubbles at the beginning or end of the curve if you plot it on a graph. Usually it means that something very interesting is happening that is out of the ordinary and could provide valuable intelligence to the data analyst. That information is not apparent from watching the big stream of machine generated data go by.

So how does a machine actually learn? The old way of doing things was to store each piece in a database and then try to look it up. It was like going to the library, and reading every card index of the subject matter of the information that you are trying to look up.  Needless to say, it doesn't work very well if you have millions of cards to go through. There had to be a better way, and that better way was the artificial neural network.  It is the basis of machine learning.

An artificial neuron is a very simple thing, and is quite stupid actually. All that it can do, is add, multiply, compute just one math function (a formula)  and compare the result. However this little virtual, self-learning thingie is the basis of all machine learning. You can gang hundreds and even thousands of them together in a massively parallel system, and they can do very complex things like recognize faces and handwriting, find doorways for robots, and tease out the latest trends in footwear.
This is how it works. Let's suppose that you want to teach your machine to recognize the number 42, which according to the "Hitchhiker's Guide To The Galaxy" is  the answer to the Ultimate Question of Life, the Universe, and Everything as computed by the Earth which is a huge organic computer.
You could do this with the simplest example of an artificial neural network. It is a single neuron consisting of an input and an output.  All of the knowledge of recognizing the number 42 is stored simply as a number, in a value called the weight. And no, the weight is not 42.  The weight is the numeric value that determines if the neural network hoists up a flag indicating that it has seen the number 42.

There is another hidden input number that is unchanging in value for all inputs, and it is called the bias.  The bias is like a control number. A very simple analogy, is that it is like a thermostat. In real life, a thermostat controls the range at which a furnace will fire. In an artificial neural network, it has the same function. It determines the range at which the neuron will fire to indicate the number 42.
So when you present a any number to the input, the neuron takes it and multiplies that number by a weight. It also multiplies the bias by a weight. It adds the two together. Then it spoon feeds the number down the chute into the activator. This is a go-no go threshold. The activator consists of a mathematic formula that defines a function.   It puts out a number between zero and one. This activation function is very unique in the fact that no matter what number you feed into it, it always gives the answer in a very long decimal from zero to one. It is like a thermometer. The closer to the right answer it gets, the closer to the value of 1 comes out of the activation function.  If the answer is less than one, it is a failure or a no-go. The neuron doesn't fire.

You don't even have to determine the weight. The neuron can be trained.  The training is called back propagation. In the training mode, you show it a whole bunch of numbers called a training set, and when the input number is 42, you ask the neuron to indicate the right answer by responding with a value of 1.  Any other wrong number will show a zero at the output.  When run it, and it gets the answer wrong, it adjusts the weight a little bit and tries again.  You keep running the training set until it knows the right answer.  It is that simple.

What makes this a powerful concept, is that you can gang hundreds of neurons together, and machine learning can do quite complex stuff. Behold the amazing Dark Arts of a Grade 10 math book.

How To Be A Billionaire Using Big Data and Machine Learning in Three Easy Paradigms

1) Download WireShark and load it onto a laptop with the biggest hard disk storage that you can find.
2) Go to the airport and sit there all day using the free airport WiFi, and turn on the record function on Wireshark
3) Use data-mining and machine learning on the datasets.

The billion dollar platform idea will emerge from the data. Guarantee it.

A Start to Artificial Conciousness - Making A Computer Worry With Machine Learning

Bring things spring from small seeds. This is the thought that keeps running through my mind when I think of Artificial Consciousness in computers. Ever since I saw the Imitation Game and the story of Alan Turing, I wondered how such an intelligent man could think that computers could think.  Of course he stipulated that the thought process was different, than in humans.

Then along came Dr. Stephen Thaler who artificially introduced the idea of perturbations in computer "thinking" and got a patent for it. A perturbation is essential to artificial consciousness.  Essentially, a computer is programmed to linearly follow an execution path of its program. Even in artificial neural networks, the output of one perpceptron is fed into another layer.  It is a linear defined path.  Thaler introduced perturbations by selectively killing off perceptrons in a layer and in what he describes as artificial neuronal near-death, and the machine becomes a creative design machine.  His landmark example was a neural network that identified coffee cups and when it was brain-damaged, it came up with creative coffee cup designs.

Perturbations can come from all things. They can come from random events. But in the state of consciousness of any sentient thing or being (notice I now have to add things, because computers have the ability to become sentient), perturbations can come from the state of consciousness itself.  A prime example is worry. We observe something within our conscious sphere, and we think about it, make a judgement about it, add the judgement to the thought process, and keep recycling the thought in an obsessive compulsive state, and you have what is known as worry.

Going back to the opening statements of big things spring from small seeds, the thought struck me that I could make my computer worry.  It would be a small worry program to start with, but then I could hop it up to another layer of abstraction and make a universal computer worry module that could become part of the Operating System.  It would be the Worry Service.

Here's how a simple version would work. The Worry Service runs a task manager, a memory monitor or a CPU usage monitor in the background.  The minute that it detects that memory or CPU is approaching 100% or saturation, it kicks the worry.exe module. The worry module essentially assumes the highest thread priority, prints out on any display saying "I am incredibly busy" and deallocates processing priority to the heavy task, slowing it down.  It then detects that the task is slowed down, and kicks another worry module about its lack of performance.  The worry modules are able to be queried, and their response is always "I am incredibly busy and it is affecting my performance".  The worry module also writes to every log that it can, and using a machine learning neural network, reinforces the worry parameters so they automatically fire at lower thresholds.  Once the busy task is completed, the worry abates and slows down, and the computer becomes efficient again.  Of course, if the machine-learning neural network is too effective and eager in kicking in the worry, it becomes a compulsive worry, and needs to see a programming computer psychiatrist to up the thresholds of its worry mechanism by running a few positive reinforcement training epochs.  All of this technology is available now.

But I can just see it. Some Goth programmer will chain all of the worry modules into the depression module, making the computer virtually worthless for sustained work.

The very first step to scary artificial intelligence, is making a computer with the ability to navel gaze. This is a start. I am convinced that human consciousness is merely an accident of an over-developed tropism, and the evolution of Artificial Consciousness can start with this simple step -- a computer worry wart. Windows machines will be the worst worry warts and the most depressive among the conscious computers.

Preventing The Pilot-Going-Nuts Syndrome with Machine Learning & Remote Control

We have a new thing to worry about in the skies. In the 1950's, it was airline crashes because metal fatigue and cabin pressurization was not that well understood. In the 1960's, we had the airline celebrity extirpation phase, taking out stars like Buddy Holly, Richie Valens, the Big Bopper, Patsy Cline, Jim Reeves, Otis Redding and boxer Rocky Marciano to boot. In the 1970's we started having hijackings to Cuba, and terrorists attacks that continued until the present day.  We have had underwear bombers, crashing planes into skyscrapers, bombs R us, ground to air missiles and all sorts of imaginative ways to take airliners out of the sky.  And now we have a new threat in the skies -- The Pilot Going Nuts Syndrome. It probably happened on Malaysian Airlines MH370 and we have the GermanWings pilot deliberately crashing into the Alps.

The Pilot-Going-Nuts Syndrome is totally stoppable with Machine Learning, Artificial Intelligence and a bit of remote control.

For example, we now have drones taking off from Alabama, flying to the Middle East, blasting a terrorist in his tent upwards to meet his Allah in pieces and collect his virgins  in some godforsaken place and then the drone flies home while the operator is eating a pulled pork sandwich somewhere in a bunker near Huntsville (irony in the name as well).  So the ability for remote control is well established.

Now let's take machine learning.  After an airplane flies a route from Barcelona to Duesseldorf ten times, any machine-learning program knows the flight plan by rote. Even Microsoft's Azure platform in the throes of the Blue Screen of Death, is smart enough to learn that route.  So if you embed a program like that into the avionics, and you input the flight plan in as well, any Pilot-Going-Nuts sufferer could be thwarted.

The minute that Co-Pilot Cuckoo For CocoaPuffs tries to take the plane off autopilot in direct deviation of the flight plan (especially at cruising altitude), the smart avionics notifies Air Traffic Control and asks for a OK semaphore.  In the meantime, the computer says to Lieutenant CocoaPuffs "I'm sorry Dave, I can't let you do that". In case of real emergency, any major deviation to the flight plan could be done using biometric authentication of both pilots like fingerprints or iris scan.  Having both pilots go crazy is a huge longshot, unless their turbans are made from the ISIS flag.  The double authentication is that if  one pilot uses the loo, the plane and passengers are not sh*t-out-of-luck if the other one takes leave of his senses.

This is so easy to fix, I don't know why its not a slam dunk. The only reason that I can think of, is that taking a chance with batsh*t crazy pilots is a lot cheaper than refitting aircraft with anti-crazy avionics.  However the system will pay for itself by preventing just two airplane annihilations by apesh*t crazy, maniacal sky jockeys.

Self-driving cars are first, and self-flying planes will be next.  And twenty years later, we will even have self-cleaning toilets.  It will follow the same trajectory as we put a man on the moon in 1969, and luggage never got wheels until the late 1980's.  In the meantime, tonight I am painting huge Rorschach ink blots on my carry-on luggage, and if any of the flight crew gets googley-eyed or starts to salivate when seeing my luggage, I am getting off.  The momentary upgrade to first class when the plane hits the mountain is not worth it.

Future Job Category ~ Data Grader And Goal-Oriented Valuator

The one thing that I have learned from designing a remarketing platform, is that there is a market for absolutely everything.  This was exemplified by my trip to Siberia. Near Finland's border I discovered a Russian millionaire who made his money from crap. Literally.  He was a chicken farmer.  Up on the north peninsula to Finland, the ground is rocky and devoid of nutrients.  Chicken crap is an amazing fertilizer full of nitrogen.  He would trade piles of chicken crap for used Swedish cars - mostly beaters that the Finnish farmers bought cheaply.  Getting a car in Russia is tough if you are not in a major center, and he marked up his cars considerable and within a few years, became a millionaire.  Like I say, there is a market for everything.

I was watching a video on the modern fur trade, from the trapping right down to the auction. It was fascinating because when we think of fur trade, we think of glamorous women in fur coats on the runway.   In the case of furs, there is an industrial market for crap furs.  The Chinese are the biggest buyers. They will buy rabbit fur, and furs that are not good enough for clothing, and use them for toys, novelties, lining on the inside of boots -- wherever.

So there is a job description in the fur purveying process called a fur grader.  Before the auction, he and his staff go through the bales and bundles of furs, and groups them according to quality and type. Bales for sale consist of similar types and grades so that the quality of the bale is consistent.

As I was watching this, it struck me that the same thing will happen to data.  There will be a data grader who will assemble datasets, grade them, evaluate their marketability, cleanse the data, and put it on a data exchange.  In the case of valuable datasets, there will be a data auction.  And if you are thinking of starting a data exchange, have I got the platform for you !!!

You will also get the arbitrageur and day trader of data. If I see I dataset that is going cheap, I just may pick it up.  You see, I will have machine learning on my side, so I can process it.  I will do goal-oriented mining, and create multiple datasets of differing things with differing values.  Much like capital partners buy and ailing company, and break up and sell its parts for more than the overall thing was worth, the same will be done for data.

It's a brave new world out there.  Folks in the industry used to think that the never-ending, endlessly multiplying streams of data were a scourge.  They are actually a raw material and an asset. After all, there is a market for everything.  Sign up for my emails on the right for more.

# DO-IT #tag for NLP machine learning for this article.

Big Data, Data Mining and Machine Learning in Advertising

The advertising industry has been pretty much on the ball when it comes to exploiting audiences and corporations in the name of spreading a brand message. However Google has eaten their lunch when it come to advertising revenues and the sole reason is that Big Data is the most effective way to target a demographic, and Google is the king of Big Data. Google knows what people want to buy from the searches that the consumers do. They have the stats on what makes someone click, when they click, what best to make them click, where they are predicted to live and pretty much everything that an advertiser wants to know to reach their audience.

The laggards in the industry are the big and small advertising agencies and brokers. I can predict that the biggest, most profitable players that will be big in advertising in the next 5 years just by examining who is adopting data mining, big data and machine learning in the advertising field.

Advertising has been bought and sold using a very large degree of granularity. For example, its a slam dunk that if you want to get a brand message across, the quickest way is to buy a Super Bowl ad. But not everyone is a Fortune 500 company that can afford an ad in the ... the ... well .. Super Bowl of Advertising.

So how will advertising agencies monetize themselves and create huge and diverse revenue streams in the very near future?

Here is an example of how it will work. With Big Data and Machine Learning, the advertising industry can operate like a futures exchange for media placement. They will craft the brand message. Big Data and machine learning will not only tell them what the most effective demographic will be, but it can do it with a fine degree of granularity as well.  Advertising will be more goal oriented. If an advertiser wants to find new markets in a new demographic, Big Data and Machine Learning will tell the advertiser where to do it. It will also outline the optimal time, the optimal vehicle, the optimal message and the optimal leitmotif of the brand message. If they want to expand their market penetration with their current target demographic, they will know where to do that as well.

Finer degree of granularity will mean real time media monitoring with a control room full of Bloomberg-like stock market screens. If Reddit is trending with a million hits per hour with a content classification linked to the 25-35 year old demographic, it may be time to buy a spot from a ad placement futures database and pop the content onto the site within a couple of minutes. Advertising space will be bought in bulk and traded in smaller elements like a commodity exchange in real time.  It is media-adaptive advertising.

If CNN has an exclusive breaking story and millions are flocking to their site, the viewing demographic can be machine-analyzed in real time (a Bayesian probability, again gleaned from millions of training epochs) and a highly effective ad can be placed that will have an instant ROI or return on investment that would be stratospheric compared to a regular buy.

Amazon or Google will develop a real time ad engine platform (RTADS --- or they will buy one from guys like me), and it will be the next big thing. App developers will be paid to incorporated RTADS (Real Time Ad Delivery System) so that trending, topical media events can be exploited with embedded, dynamic advertising based on who is consuming the media.  As a matter of fact, the next Bloomberg millionaire will be the optimal ad monitoring station for advertising insertion. It is a function that is unknown today, and tomorrow the developers and maintainers of the system will have a coherent job description for it..

Machine Learning can predict good wine years and classical vintage. It can predict which films will win the Oscars. It can guide robots and munitions. Once the advertising agencies see the harnessed power of Big Data, Data Mining and Machine Learning, they will convert quickly, and eventually those players operating in the old paradigms will die. It is only a matter of time.

It's a great time to be alive if you are into Machine Learning, Artificial Intelligence, Data Mining and Big Data.

How To Be The Next Big Data, Machine-Learning Millionaire (in 3 Easy Paradigms)

I have a special talent. I make other people rich -- extremely rich! The first time that it happened, was in the early 1990's. I invented a new type of golf tee. The lawyer that I hired patented it under an umbrella proxy over which he had power of attorney. He said it was necessary for the financing of it. It was a long story, but I never saw a dime. He is retired in Turks & Caicos. It was particularly painful to find one of my designs on the golf course, now that the patent is expired.

The second time, I made a pile myself. It was during the tech boom, and the tech crash took us out with the speed of a tsunami. The third time was when I was when I was consulting as a technical architect to a G8 government. We were sitting in a scrum, and one of the team members mentioned that the telecom giant Nortel was trading at .75 cents a share. A few short months ago, it was at $130 per share. This team member said that it might be worthwhile to throw ten grand at it.  I said "yeah, yeah, lets do it" and ultimately forgot.  A young programmer on our team, believed in my endorsement of the stock and threw much more than that at it.  He got out when it reached $16 a share. Do the math. Nortel eventually collapsed, but our intrepid friend made such a pile that he bought a BMW and never got out his pajamas for the next few years.

The last time that I made someone rich, was that I was in idle conversation with an elderly Manhattan-based writer last May (May 2014). He was a meditating Buddhist who lived simply and had a pile of cash to bet on the next big thing. He asked me what the next big thing would be. I told him that it would be the Internet of Everything.

He asked me who would be the big player in the Internet of Everything. I told him that Sierra Wireless (stock symbol SWIR) had foundation patents and had the potential to be the next Google or Apple. Since May ( a short 9 months ago) he has doubled his money. He thinks that I am genius. Needless to say, I didn't get on the ride with him.

And now, you too can benefit from my largess and become a millionaire in the field of Big Data and Machine Learning. You can do it in three easy paradigms.

Paradigm 1: Write a Universal Lightweight Data Inter-change Universal Sensor Data Transfer Protocol. Use JSON or XML. It is dead easy. And actually, you don't have to do it. I did it for you in this blog entry! And for the ultra-lazy putative millionaire, here is an example of it:

For that, I propose my handy-dandy XML based Universal Sensor Transfer Protocol, but instead of XML it is STML or Sensor Transfer Markup Language.  Here is what it looks like:
(quote)
<?stml version="1.0" encoding="utf-8"?>
<sensor>
      <name>Caliente Temp Sensor</name>
      <serial_no>000-000-001</serial_no>
      <units>degrees</units>
      <scale>Fahrenheit</scale>
       <reading>65.9</reading>
        <timestamp>22/10/2014:20:26</timestamp>
</sensor>

Paradigm 2: Use some open source stuff like Apache Tomcat, MySQL and open source stuff to write a RESTful service to pop all of the sensor readings into a database.

Paradigm 3: Using your favorite machine-learning platform, input the data and train the living crap out of the data, preferably in real time to make ultra-smart houses, ultra-smart factories, ultra-smart utilities, etc etc etc.  Everyone will want one of your platforms, because the system will be fire-up-and-forget as the military guys say of intelligent systems.  The machine will learn what is normal, call someone when it ain't, and send back feedback to optimize whatever the sensor controls and make life, smarter, easier and better. It will save everyone time, energy, human work hours and time.    AND IT WILL MAKE YOU FRIGGIN' RICH.  Everybody will want one of these systems.

And here is the disrupter idea for the disruptive idea:

The cutesy coder guys will offload the training to the cloud and push the results to a smartphone.

There you go. You are welcome. This idea is a sure-fire winner to make you a millionaire.  I would do this project, except that I am too busy with being Chief Technology Officer of our company.  Also I am working on a recreational pharmaceutical company with a new designer drug offering. We are combining birth control pills with LSD so that you can take a trip without the kids. Oughta be a slam-dunk as well!

Oh, and be sure to sign up in the box to the lower right for my occasional non-obtrusive emails with further app ideas, cogitation on Deep Learning and AI, and futurism thoughts on tech. There will be a few monetizable ideas there as well.

Conquering The Time Domain in Marketing With Big Data & Analytics

Our platform sells big ticket items -- it remarkets and wholesales used cars.  The supply chain is well defined. A new car dealer takes in a car on trade. He really doesn't want to do it, because most used cars are not moneymakers. If it sits on his or her used car lot forever, it loses money for him/her  instead of making money. That is because the new car inventory underlying that trade-in is usually financed.  To complete the deal cycle of used car trade-in -> new car purchase -> used car sale for recouping money, the used car has to sell quickly.

Secondhand car dealers in small markets are experts at what sells and for how much, and what the market is willing to pay. They have intense local knowledge of their geographic domain.  A lot of the time, new car dealers do not have that expertise and/or knowledge.

Coupled to this fact, is that in spite of the parameters of make, model, year and condition, there is no uniform valuation for a used vehicle. It varies by area, time of year, color of vehicle, geographical location, local economy and a million and one different factors. Folks like Black Book try to standardize the valuation for the process, but at best, they are only a rough guide based on auction prices around the continent.

As we have shown in this article, the Black Book paradigm of gleaning value from auctions is not  accurate because up to two-thirds of all vehicles are remarketed through relationship-based wholesaling, and never hit the auction floor.

Coupled to that, there is no "real price" for any used vehicle. What a vehicle sells for is based on what the new car dealer has in it (a combination of what he thinks the vehicle is worth and the discount that he has allowed on the new car that was bought with this trade-in). A good example of this is that on our platform recently, a dealer had $9,000 in an SUV. That's the reserve price that he put on his vehicle, because that is what he needed to make the deal profitable. He let the market forces dictate the ultimate price, but he needed $9,000. The SUV sold for $27,000 in the fair and equitable marketplace on our platform. So what was the vehicle worth? It was worth $9000 to one person and triple that to another. This is why we introduced crowd-sourcing valuations into our platform.

But there is one other element in marketing that transcends specific sectors, and that is the time element.  Currently, a light manufacturer will do a run of product, and try to flog it off to wholesalers, retailers, online markets etc.  It costs money to hold the product in inventory.

Technology such as 3D printing and print on demand for digital books alleviates some inventory build-up, but generally the time domain is huge in merchandising and marketing.  What I mean by that, is that inventory is built up, and disposed of over time at ever-changing prices based on supply and demand. There is a measurable, considerable cost to storing inventory.

As pointed out in the automobile remarketing industry that we are in, the domain of time is a negative one. The longer an item stays in inventory, the less it is worth, and the larger the drag on the bottom line. Positive revenue stream is based on timely sales.

To conquer the time domain, we used Big Data to our advantage. We coupled it with our relationship-based sales paradigm described in the above link, and as it turns out, the piece of technology was patentable, and we have foundation patents pending in that area.

This is how it works. The whole idea is to move inventory quickly. We have mapped the buyer/seller network relationships (a social network media type of construct) with trusted buyer zones based on previous commercial relationships. This is the first step in the process that we have created.  The product is offered to this trusted network group for a limited time (in our case, four hours is a norm). If the product does not sell, what then? As the clock ticks, money is lost.

The second step involves Analytics.  We use Big Data to find in our customer base, and in other databases, who is the best and most frequent kind of buyer for this product. The machine assembles a top-10 list based on a proprietary algorithm of sifting through Big Data, and offers it to that ad hoc group of buyers for a limited time.  The really nice part, is that once buyers find out about the top ten, we have a potential revenue stream where they will pay for early market information and a chance at a deal.

When that time expires, the platform has the smarts to move the inventory to the next phase of selling. In our case, it goes to general auction to the open group of buyers, and if that fails, the platform has the technology and ability to transfer the inventory to a classified type of listing.

Our competitive advantage, is that we have conquered the time domain with relationship-based social network selling for the first step, and the use of Big Data for the second step. Our competitors use the third step as their first step.

Big Data has a huge advantage in conquering the time domain.  Suppose as a manufacturer, or even a retailer you had a platform to sell all of your inventory in a specified time-frame. With a platform such as ours, adapted to other fields, you could commoditize your inventory, and using relationship-based selling coupled with Big Data, you could have your inventory dispersed just as it was about to leave the factory floor, or arrive on a shipping dock.  Big Data will even tell you how much inventory to order and make.

Merchandising and selling will all change drastically in the next few years, and those that don't adopt the Analytics/Machine Learning paradigm, will bite the dust.

An End To Dangerous Big Data Stalking

You are being stalked. Every website that you visit may add a stalker in the form of tracking cookies to your browser. They know where you have been.  And with just a modicum of inference they know who you are.

This web tracking is pervasive. It all goes into a big database. If for some reason, you enter your name on a form, and the form is transmitted to the website in what is known as an HTTP Post, they will harvest your name. But even without your name, they will know what demographic you belong to. They will know your financial standing and how much you earn. They will know what music you listen to and what clothes you buy. And all of this information is processed without the benefit of human eyes sorting and classifying this data. Machine Learning is pervasive.

But here is what is most dangerous about these stalkers.  They can make the wrong inference, and put you on a watch list that may be impossible to get off, or you may not even know about.  Here is a scenario that could make you a terrorist according to Big Data and Machine Learning.

You are sipping your morning coffee looking at Facebook, and you see a heartbreaking picture of a child caught in the clutches of war in the Middle East.  You "Like" the photo.  Then it is time for you to go to the airport. You are flying business class and are given a choice of food. There are Halal meals. You are an adventurous foodie, so you tick it to try it.   Coupled to that, is that you have an aisle seat.  Then you check your Twitter feed.  Someone posts about "Freedom of Religion",  You favorite the tweet. In the business section of a European website, you see the add for a hedge fund that promises great returns. You click for more information.  What you don't know, is that you have put the Big Data Digital Stalkers into overdrive, and you are now a person of interest to several agencies.

As it turns out, the photo that you "Liked" was posted by a terrorist group to garner sympathy.  All of the "Likes" are collected as possible links to these terrorists. You are in another database because you chose Halal food instead of the bacon cheeseburger.  The aisle seat is problematic. Hijackers do not take window seats.  The "Freedom of Religion" tweet was sponsored by the Muslim Anti-Defamation League. Into another database you go.  The hedge fund promising great returns is headquartered in the Cayman Islands. The IRS is suddenly interested in you.

The most dangerous thing about Big Data Stalkers, that that they make Bayesian Inferences which are probabilities.  Probabilities are just that. They are not certainty. Even with a 99% probability, the next event in the sample space could be wrong -- not what the probability predicts.  Machine Learning and Big Data Stalkers are a clear and present danger to personal privacy.

The other intrusion on your life from Big Data Stalking is the stuff done with commercial enterprises. They aim to learn absolutely everything they can about you, because they can sell that data.  Big Data can produce new or enhanced revenue streams.  Is there a way out of this?

I say that there can be.  With a paradigm shift, the consumers of Big Data can get what they want, and your privacy can be protected. How you ask? With a little dash of technology.

Let's suppose that you turn the tables and consent to limited data tracking. That data tracking is now bowdlerized, meaning that sensitive personal stuff is obfuscated or removed. This is done by an app on your device, cell phone, tablet or computer.  Then you are paid for that data to the highest bidder.  Everyone is happy, and you the consumer benefit from the data collection.

As for the other stuff, technology can help too.  I am a huge proponent of Artificial Intelligence.  Suppose that you had a proxy entity digital assistant called Blocker.  Blocker would surf the web for you, executing your Likes and Dislikes while retaining your anonymity. Blocker would run on a proxy service, so that even IP addresses would be hidden. On top of that, it would surf in anonymous mode.  If there wasn't any personal user data to be had, your privacy would be protected. The data flow wouldn't entirely be impeded because through content analysis, you could still make pretty good inferences of the humans behind any wall. For example, a grandma living in Norway wouldn't be listening to rap music, but her grandson might be.

So, with a bit of different thinking, we can mitigate the dangers of Big Data Stalkers. The unfortunate thing, is that many denizens of the Internet, do know or don't care about the Stalkers.

Impressed with Microsoft -- finally -- Azure

I, like a lot of other geeks, have become greatly disillusioned with Microsoft in the past several years. I saw them as anti-innovative, fat-cats protected a revenue stream that did not favors for its users, and becoming a stodgy, quaint grandparent in a tech world, where it thought that it was still the same sex object that it was in its early daze.

Microsoft, in my opinion has hung on too long to its archaic operating system which is essentially one big kludge onto top of a stack of turtles of kludges all the way down to the bare silicon. All of their innovations in almost every endeavor from tablets to phones, to music services,  have been market failures because they stubbornly resisted changes to their bloated, digital-cholesterol clogged operating system.  If they truly want to be innovative, they would ditch it in favor of a brand of QNX or Linux for a sleek, less vulnerable system.  Back in the early daze of the 8086 microprocessor, I saw a QNX system being able to boot from one floppy disk, and in its day, that was amazing.

Now that I got that off my chest, I must grudgingly admit that Microsoft has lit a spark that impresses me with their Azure big data suite.  If they are going to re-invent themselves and breathe new life back into the corporation and become innovative again, then Azure might be the vehicle.

Big Data is where it is at, and where it is going to be if we want to manage and monetize the Internet of Everything. And Microsoft Azure is trying to create and promulgate products to that end with Azure.  I only became aware of Azure when several members of the Azure team followed me on Twitter, and when I checked them out, I realized that it wasn't Bill Gates' Microsoft. I really liked what I saw.

Azure offers data analysis as a service, and they have a free component. It is done in a quasi-cloud environment, and from what I see, once you graduate from the newbie class, the prices is okay.  The good news is that there is a link to some pretty nifty free tools.  Here is the link:

https://datamarket.azure.com/browse?query=machine%20learning&price=free

The tools are varied, useful and intriguing.

Microsoft just may have a chance to dominate the market.  Their thin edge of the wedge with azure is great, but they must follow the template of Microsoft Word when it started to dominate the marketplace.  Back in the day, personal computers were useful, but not that useful when it came to creating documents electronically.  The IBM Selectric typewriter was the weapon of choice to use up reams of paper.  Then along came the word processor.  Dr. An Wang made a fortune from inventing computer memory, and then sunk his money into Wang Labs headquartered in Lowell Massachusetts. The Wang word processor became ubiquitous for several years. It was a dedicated piece of hardware, and tightly coupled software that didn't do anything else except create formatted documents.  Prior to that, electronic documents were printed on a dot matrix or impact printer without stylings.  (The Wang OS was the first OS that I successfully hacked).

Microsoft Word came out and essentially destroyed the word processor.  It was order of magnitude cheaper, easier to use and a mere fraction of the cost.  It is still the dominant document creator to this day.  Microsoft needs to do the same thing with Azure.

Right now, a lot of the Azure products use the statistical language R. Other plugins calculate linear regression, and all sorts of stuff like standard deviation, blah blah blah.  Microsoft needs to hide that under a big layer of abstraction and make all of that invisible to the end user. Picture the end user who runs a niche cafe in a hip town. Their Point-Of-Sale and computer system collects metrics, meta-data and machine data.  The owners of this data has no idea what this data can tell them or how they can increase their revenue streams.  They don't know Bayesian inference from degree of confidence.

Microsoft needs to build data analysis for the common person, like they built word processing for the common person.  If they do that, they will take their company into the next century. If not, they will be the biggest Edsel of the tech industry.  However, for the first time in a long time, I like what I am seeing come out of Microsoft.