Connected Autonomous Cars, Big Data, and Not Re-inventing The Wheel

Smart Roads Need Not Be So Smart

The introduction of technologies into daily life lets us let go of old paradigms and ways of doing things. It also lets us jettison conventional ideas. I was in a deep conversation last night at dinner with a philosopher friend and I was telling him that I was working with automotive blockchain as a true ledger -- especially for self-driving cars. I mentioned that perhaps we would need smart roads or smart road sign sensors to indicate things like speed limits and such to the autonomous car.

We got into a discussion on how self-driving cars will change everything about mobility -- even the concept of your car sitting in a parking lot all day. For example, after your self-driving car drops you off at work, you can send it out to work for money as an Uber car, and it comes to pick you up after your work day is done. Or you can send it home.

My friend opined that with this and other technologies, one is only limited by the imagination as to what can be implemented. He didn't think that we would need smart roads. He pointed out that using Big Data, the computational load of self-driving cars could be significantly reduced. We wouldn't need smart roads hardware embedded in geographic locations. It was brilliant.

Here is how it will work. My blockchain is intended as a vehicle black box recorder. Everything with the connected car is recorded in real time. This includes GPS coordinates, date, time, and all of the instructions issued by the operating system of the vehicle to drive a particular stretch of road. Here is the clever bit.

Suppose all of this stuff is uploaded to a central repository, and is searchable. The connected autonomous vehicle, upon entering a specific roadway, would access this information. Through Big Data analytics, it would now know average driving conditions and speed for time of day, season of the year, rush hour, rain, sleet snow and it would know the salient features of the roadway. For example, you won't have self-driving cars running red lights or stop signs like you see on Youtube now, because you will have those features available to you. It will know things like where to watch out for other vehicles exiting a driveway (based on history of cars stopping to let these vehicles out). In other words, you will have a smart roadway without sensors and without Internet of Things (IoT) indicators. It will be like Google Street View for autonomous vehicles. The vehicles will be able to search, find and download roadway features, and use these features to navigate, without intense computational load on the car operating system. The onboard driving system would have to only detect anomalies and other traffic. You would not be re-inventing a computational feature map every time that you went down that road.

Smart roads would be smart because there would be a driving-instruction history created by thousands of vehicles on how to navigate these roads. They would be mapped with a GIS system that included driving parameters.

It would be the Google search engine for the brain inside your car. I am sure that Google has already thought of this concept. They were forward-looking enough to start Street View, but there is always room for a better mousetrap hatched by a disrupter.  The disruption in this case, is to present the driving parameters in a way that will be understood by all self-driving cars. Therein lies the next billion dollar play.

The Caine Mutiny & Self-Driving Cars

One of my most treasured books is a first edition of Herman Wouk's novel "Caine Mutiny".  Published in 1951, it won the Pulitzer Prize and it was on the New York Times Bestseller List for 122 weeks.  I was pleasantly surprised to find at this time of  me writing this blog posting, that Herman Wouk is still alive and 100 years old.

The movie version of Caine Mutiny starring Humphrey Bogart (pictured above) is pretty cheesy, and didn't give a tenth of the subtext that the book gave.

Briefly, the story takes place during World War II in the Pacific naval theater on an obsolete Destroyer Minesweeper called the USS Caine.  Since Wouk himself served on two of these types of ships, and his naval career followed the trajectory of an officer in the book, the level of detail of running a naval warship is stunning and intricately documented.

The story line goes like this.  An ineffective, incompetent captain is assigned to the ship. His name is Queeg, and in the movie he is played by Humphrey Bogart.  Queeg tries to mask and overcome his incompetence by becoming a martinet, blaming his crew for his mistakes and covering up acts of cowardice in battle.  His behavior gets weirder as his commanders dress him down for his mistakes while commanding the Caine.

During a typhoon, an able, but not an intellectual executive officer named Maryk countermands a  Queeg order to maintain fleet course that would have sunk the ship.  The ship would have floundered by sailing with the wind and sunk in the high waves.  Instead Maryk saves the ship by correctly heading into the wind face on to last out the winds and waves using the powerful engines of the destroyer to hold his position in the most stable way possible.  As a result of the action, some of the crew is charged with mutiny, and the denouement is the court martial trial.

In the book, the author details that minutia that enables reservists like shoe salesmen, factory workers and college students to conduct naval warfare on a historic scale with competence that rivals the great sea battles such as those fought by John Paul Jones and Lord Admiral Nelson.

So what does this have to do with AI, and self-driving cars? This is a James Burke Connections moment. In the book, the protagonist is a young, rich, immature ensign and former playboy named Willie Keith. Through the course of the book, he matures as a person and rises through the ranks to eventually become the captain of the USS Caine.  As a young initiate and a commissioned officer, he is assigned OOD or Officer of the Deck. He learns how to navigate, calculate courses, issue commands that keep the ship and fleet zigzagging to avoid submarine torpedo and maintain a sonar net around the fleet. He is the captain's representative when the captain is not on the bridge.

As OOD, Willie has to sign the log book at the end of his watch, and the first time that he signs it, it is a single notation: "Steaming as before. Willie Seward Keith USNR". This was highly instructive to me as it was a Eureka moment for my thoughts on self-driving cars, and the whole reason why I brought up the Caine Mutiny in the first place.

The author, Herman Wouk was quite accurate in the mundane details of how the ship is run, and what it did while sailing in convoy, and what he had to execute as his own OOD duties when he served about the USS Zane and the USS Southard. The log would record the course that they were sailing, but would not record the zigs, zags, changes of course, the swapping out of positions for the sonar submarine screen with other destroyers in the convoy, and other "housekeeping" details.  These were not logged.  But the important events were, and the navy would eventually review those logs if something extraordinary happened. The logs would hold lessons for a postmortem analysis.

This was the "aha" moment for me.  Self driving cars need to log all of their journeys, all of the time in a temporal database.  An explanation of a temporal data base is here, but essentially it records what was true at any given time.  But to store redundant information that belongs to the Department of Redundancy Department is not necessary.  If you embarked on a trip on the interstate, you would record your start time and your end time.  If you didn't stop at a rest stop, or get into an accident, or do anything to impede your progress,  that is all that you need to record.  You don't have to record a lot of data at every minute.   The biggest inference that you can make, would be like the log book of the USS Caine - "Steaming as before.".

But why bother to record anything including the status quo, if you are steaming as before? Why bother keeping logs?  Because of the rich knowledge buried in the data.  From that small bit of recorded information, you can derive your average velocity.  From the collected data of many, many trips by many cars, you can infer a lot of things.  You can infer if the weather was bad if it took a time slower than normal and you didn't stop anywhere. You can infer mechanical conditions. You can infer traffic density. You can engineer a lot of features from the trip data.

But the most valuable data gleaned from collecting a lot of logs of every trip that a self-driving car makes, is that you can make a self-driving car a lot smarter, if it and others like it can learn from the mistakes and successful corrective actions of other vehicles that have driven the road.

Driving is a not a complete linear event.  Speed varies. Road conditions vary. Random events take place such as an animal runs out in front of the car, or another vehicle cuts you off.  If you collected every IRRR (Input, Reaction, Resultant Reaction), you would have quite a library of events that could only improve the driving ability of every autonomous car.  Self-driving cars could emulate human drivers. They can get better by more practice and by learning from others.

But lets go one further. In a connected car, the internet is always on.  The car is always positionally aware of where it is.  Suppose that as you drove down a particular stretch of road, an electronic milestone would download some neural nets into the car that would enhance the car's ability to navigate that particular section of road.  And where would that come from?  From the collected event logs that not only had the "Steaming as before" data but the data on how other drivers successfully negotiated conditions and hazards on that particular stretch of road.

With storage being incredibly cheap and in the cloud, this sort of continuous data collection and processing would be easy to collect.  Many people will point out privacy concerns, but if there was a standard for the neural nets or a library of driving moves for self-driving cars or even a standardized neural net interchange network, only the data itself after it has been processed is required without vehicle identification.  As a matter of fact, when the legislation for self driving cars comes in, it should specifically prohibit vehicle identification in the data collection stage.

Self driving cars collecting temporal data can have lots of benefits.  With aggregated data, you can have expected MTBF or machine-determined Mean Time Between Failure.  The car could drive itself to get serviced when it is required -- usually after it has delivered you to work.  And it can book the service appointment on the fly when it reaches a critical juncture.

With the "Steaming as before" logged data collection, the act of getting driven somewhere could and will be automatic, without a thought given to the process of actually getting yourself to your destination. The car will take care of all of the logistics for you.

As for me, to hell with the data.  I'm going to invest in a connected car wash specifically to clean self-driving sensors, and have an automated beacon system to lure the autonomous cars in for a wash and wax.  They will pop in and out, and continue steaming as before but now much cleaner.

AI Risk Radar For Self-Driving Cars

There is a lot in the news with self-driving cars.  Google has one. Apple is building one. Mercedes already has a self-driving transport.  Even the kid (George Hotz) who carrier unlocked the iPhone and Playstation built himself a self-driving car.

You read about LIDAR systems (Wikipedia: Lidar (also written LIDAR, LiDAR or LADAR) is a remote sensing technology that measures distance by illuminating a target with a laser and analyzing the reflected light. Although thought by some to be an acronym of Light Detection And Ranging, the term lidar was actually created as a portmanteau of "light" and "radar".) and camera systems with real time video analysis, etc.

Several makes and models already partially autonomous functions like parallel parking and lane departure systems that warn you when you leave the lane. Some cars will autonomously steer on a highway when cruise control is on.  However what is missing from the whole autonomous driving vehicle is some real brains behind driving.

When a human is driving a car -- especially a human who is a good driver and taught to drive defensively -- they always have a layer of abstraction that I like to call a risk radar turned on.

The risk radar while driving is like a an analysis and environment supervisory circuit that is not directly involved in actually driving the car, but processes meta-data about driving.

For example, a risk radar would look at the outdoor temperature and if it is freezing, it would devote some cycles to see if there is black ice on the road.

A risk radar notices that in a two or more lane highway, if you are are driving in another vehicle's blind spot, you either slow down or speed up.

A risk radar notices that you are in a high deer collision area on the highway, so some extra cycles are devoted to looking for the pair of reflective phosphorescent eyes on the side of the road.

If you had to brake heavily to avoid collision, a risk radar will model the circumstance that caused that extreme corrective action and will be comparing all environmental parameters, conditions and events to watch out for similar situations.  When one or more of those risk factors are present, it goes into high alert mode.

A risk radar can actually modify the way that a vehicle is driven.  Self-driving cars will have to have sensors like an accelerometers to detect yaw and pitch due to high winds, or hydro-planing on a wet road,  Risk radar would note these things.

Risk radar would count cars and notice if the traffic was heavy.  Risk radar will notice the speed of travel of other cars and make inferences about it.

Risk radar will have a range of characteristics.  Not only will it have a library of risks and the mitigation actions associated with them, but it will also have both supervised and unsupervised machine learning.  In machine learning, supervised learning is where the machine is given the correct result so it can self-adjust weights and thresholds to learn risks and risk mitigation actions. By the same token, unsupervised learning is where the machine infers a function from input data (through various algorithms such as clustering etc).

The biggest element of risk radar for self-driving cars, is that it must be time-domain aware.  Time-domain awareness means that it must know that certain events and actions follow the arrow of time and may or may not be caused by the preceding state. This is state of cognition with time awareness is a form of artificial consciousness (see the blog entry below), and it important in implementing a high level autonomous decision as to what algorithm will be used to drive the car.  For example, if the risk radar warrants it, the car would move into a cautious driving algorithm. If the road was icy, the car would under-steer to prevent skidding.  This would require coordination between acceleration and braking that would be different from ordinary driving.

The risk radar necessary for cars would be an evolving paradigm.  Cars driving in downtown New York would have different risk points than cars self-driving in Minnesota.  Having said that, if there were a standardized paradigm for the interchange of trained neural nets, a GPS waypoint would load the appropriate neural nets into car for the geographic area that it was driving in.

The risk radar for self-driving cars is a necessity before these autonomous vehicles are seen ubiquitously on all streets.  It really is a Brave New World, and it is within our grasp.

Guest Post: Thoughts on Self Driving Cars By Keith Oppel - Twitter: @MistaKeets

As I am sure you have seen, we are reaching a point where there will be a major disruption within the automotive industry: the self driving car. From an optimistic point of view, the automotive industry will continue as usual for a period of time (10 years? 20 years? maybe even 30 years or more). However, from a pessimistic point of view, the transition will have many unforeseen consequences.

First the good news for people involved in the automotive industry: the change can not possibly happen overnight. It will be a very slow process as the initial self-driving cars will be very expensive (as is the case for most tech when first introduced). Unless corporations fund (or governments subsidize) a major initiative to adopt self driving cars, I would imagine that for the next 10 to 20 years at least that business will continue as usual. Some rich people will have fancy self driving cars, and everyone else will continue to buy regular cars. Some may ask "what if the car companies stop making regular cars?". My viewpoint is that where there is a demand, companies will produce product to meet the demand. Many people like to drive cars, so the pattern will continue. This will also continue to drive the after market of cars (used cars, wholesale cars, etc).

The self driving car, while a nice concept, also has the potential to create a dystopian future similar to what people read about in science fiction novels. Imagine for a moment a world where only the upper class can afford the self driving cars, and these same upper class use their wealth and political influence to obtain roads/highways/freeways which are to be used only by self driving cars. The American government in particular would enjoy this idea, as they could charge a toll for these roads.

This also creates the potential for yet more class warfare. Rich people on toll roads with self driving cars, safe from these so called "dangerous people driven cars" , while everyone else (lower class in particular) continue using regular cars....abandoned technology on poorly maintained roads (as possibly the government would put more resources into the self driving routes). In a world already highly divided by class/race/religion...this is not good.

Where does this leave people who can not afford the self driving cars, especially if we get to the point where General Motors or BMW stops making cars that have a self driving feature? The used car market. What happens when big auto abandons support for older cars? I believe people will get creative. I can see a future where aftermarket car parts are made by 3d printers. Some people however will view this as a side effect of the self driving car market. One group will want the future, the "safe" self driving car with all it's convenience. Another group will rebel against the concept, and do whatever they can to keep their regular car going so they can continue driving.

Based on the article from Vox.com that has recently being going around relating to DMCA and what car companies will do about the software in current modern cars, copyright law also comes into play when considering the future of the car. My take is that copyright will not stop people. The younger generation having very little regard for copyright is my main basis for this thought. People (in particular the young) will find a way to circumvent car software and do as they wish. Perhaps a new kind of auto mechanic? Time will tell. This idealism could also be applied towards self driving cars: people will find a way to bypass future software if it gets to the point where there is no self driving mode. Illegal steering wheel modifications for the car of the future? Possible startup idea right there.

A final thought (and this may be the one you find most interesting): the Third World. At this time the pros and cons of a self driving car is a very first world problem. As you know, people in various developing nations can not even afford a regular car, much less a self driving car! There are many countries that do not even have a good infrastructure for regular self driving cars (or public transit, but that is another matter entirely). I suppose where I am going with this is if America/EU/Japan/China start adopting self driving cars, there should still be a HUGE market for selling used cars in places like the Philippines, Africa, and various parts of South America. Suddenly an out of reach "regular" car could become affordable in the developing world. Perhaps all of the used cars of America will end up being sold in places like Cambodia and Honduras? Again, only time will really tell. Affordable used cars also can be used for job creation/economic growth. Somebody would of course need to create more infrastructure to handle additional cars on the road.