Standard Artificial Neural Network Template

For the past few weeks, I have been thinking about having a trained artificial neural network on a computer, transferring it to another computer or another operating system, or even selling the trained network in a digital exchange in the future.

It really doesn't matter in what programming language artificial neural networks are written in.  They all have the same parameters, inputs, outputs, weights, biases etc.  All of these values are particularly suited to be fed into the program using XML document based on an .XSD schema or a light-weight protocol like JSON.  However, to my knowledge, this hasn't been done, so I took it upon myself to crack one out.

It is not only useful in creating portability in a non-trained network, but it also has data elements for a trained network as well, making the results of deep learning, machine learning and AI training portable and available.

Even if there are existing binaries, creating a front end to input the values would take minimal programming, re-programming or updating.

I also took the opportunity to make it extensible and flexible. Also there are elements that are not yet developed (like an XML XSD tag for a function) but I put the capability in, once it is developed.

There are a few other interesting things included.  There is the option to define more than one activation function. The values for the local gradient, the alpha and other parameters are included for further back propagation.

There is room to include a link to the original dataset to which these nets were trained (it could be a URL, a directory pathway, a database URL etc).  There is an element to record the number of training epochs.  With all of these information, the artificial neural net can be re-created from scratch.

There is extensibility in case this network is chained to another. There is the added data dimension in case other type of neurons are invented such as accumulators, or neurons that return a probability.

I put this .xsd template on Github as a public repository. You can download it from here:

http://github.com/kenbodnar/ann_template

Or if you wish, here is the contents of the .xsd called ann.xsd.  It is heavily commented for convenience.


<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified">
  <xs:element name="artificial_neural_network">
    <xs:complexType>
      <xs:sequence>
        <!-- The "name" element is the name of the network. They should have friendly names that can be referred to if it ever goes up for sale, rent, swap, donate, or promulgate.-->
        <xs:element name="name" type="xs:string" minOccurs="1" maxOccurs="1"/>
        <!-- The "id" element is optional and can be the pkid if the values of this network are stored in an SQL (or NOSQL) database, to be called out and assembled into a network on an ad hoc basis-->
        <xs:element name="id" type="xs:integer" minOccurs="0" maxOccurs="1"/>
        <!-- The "revision" element is for configuration control-->
        <xs:element name="revision" type="xs:string" minOccurs="1" maxOccurs="1"/>
        <!-- The "revision_history" is optional and is an element to describe changes to the network -->
        <xs:element name="revision_history" type="xs:string" minOccurs="0" maxOccurs="1"/>
        <!-- The "classification element" is put in for later use. Someone will come up with a classification algorithm for types of neural nets.There is room for a multiplicity of classifications-->
        <xs:element name="classification" type="xs:string" minOccurs="0" maxOccurs="0"/>
        <!-- The "region" element is optional and will be important if the networks are chained together, and the neurons have different functions than a standard neuron, like an accumulator or a probability computer
        and are grouped by region, disk, server, cloud, partition, etc-->
        <xs:element name="region" type="xs:string" minOccurs="0" maxOccurs="1"/>
        <!-- The "description" element is an optional field, however a very useful one.-->
        <xs:element name="description" type="xs:string" minOccurs="0" maxOccurs="1"/>
        <!-- The "creator" element is optional and denotes who trained these nets -->
        <xs:element name="creator" type="xs:string" minOccurs="0" maxOccurs="1"/>
        <!-- The "notes" element is optional and is self explanatory-->
 <xs:element name="notes" type="xs:string" minOccurs="0" maxOccurs="1"/>
        <!-- The source element defines the origin of the data. It could be a URL -->
 <xs:element name="dataset_source" type="xs:string" minOccurs="0" maxOccurs="1"/>
        <!-- This optional element, together with the source data helps to recreate this network should it go wonky -->
        <xs:element name="number_of_training_epochs" type="xs:integer" minOccurs="0" maxOccurs="1"/>
        <!-- The "number_of_layers" defines the total-->
        <xs:element name="number_of_layers" type="xs:integer" minOccurs="1" maxOccurs="1"/>
        <xs:element name="layers">
          <xs:complexType>
            <xs:sequence>
              <!-- Repeat as necessary for number of layers-->
              <xs:element name="layer" type="xs:string" minOccurs="1" maxOccurs="1">
                <xs:complexType>
                  <xs:sequence>
                    <!-- Layer Naming and Neuron Naming will ultimately have a recognized convention eg. L2-N1 is Layer 2, Neuron #1-->
                    <xs:element name="layer_name" type="xs:string" minOccurs="0" maxOccurs="1"/>
                    <!-- number of neurons is for the benefit of an object-oriented constructor-->
                    <xs:element name="number_of_neurons" type="xs:integer" minOccurs="1" maxOccurs="1"/>
                    <!-- defining the neuron this is repeated as many times as necessary-->
                    <xs:element name="neuron">
                      <xs:complexType>
                        <xs:sequence>
                          <!--optional ~  currently it could be a perceptron, but it could also be a new type, like an accumulator, or probability calculator-->
                          <xs:element name="type" type="xs:string" minOccurs="0" maxOccurs="1"/>
                          <!-- name is optional ~ name will be standardized eg. L1-N1 layer/neuron pair. The reason is that there might be benefit in synaptic joining of this layer to other networks and one must define the joins -->
                          <xs:element name="name" type="xs:string" minOccurs="0" maxOccurs="1"/>
                          <!-- optional ~ again, someone will come up with a classification system-->
                          <xs:element name="neuron_classification" type="xs:string" minOccurs="0" maxOccurs="1"/>
                          <!-- number of inputs-->
                          <xs:element name="number_of_inputs" type="xs:integer" minOccurs="1" maxOccurs="1"/>
                          <!-- required if the input layer is also an output layer - eg. sigmoid, heaviside etc-->
                          <xs:element name="primary_activation_function_name" type="xs:string" minOccurs="0" maxOccurs="1"/>
                          <!-- ~ optional - there is no such thing as a xs:function - yet, but there could be in the future -->
                          <xs:element name="primary_activation_function" type="xs:function" minOccurs="0" maxOccurs="1"/>
                          <!-- in lieu of an embeddable function, a description could go here ~ optional -->
                          <xs:element name="primary_activation_function_description" type="xs:string" minOccurs="0" maxOccurs="1"/>
                          <!-- possible alternate activation functions eg. sigmoid, heaviside etc-->
                          <xs:element name="alternate_activation_function_name" type="xs:string" minOccurs="0" maxOccurs="1"/>
                          <!-- ~ optional - there is no such thing as a xs:function - yet, but there could be in the future -->
                          <xs:element name="alternate__activation_function" type="xs:function" minOccurs="0" maxOccurs="1"/>
                          <!-- in lieu of an embeddable function, a description could go here ~ optional -->
                          <xs:element name="alternate__activation_function_description" type="xs:string" minOccurs="0" maxOccurs="1"/>
                          <!-- if this is an output layer or requires an activation threshold-->
                          <xs:element name="activation_threshold" type="xs:double" minOccurs="1" maxOccurs="1"/>
                          <xs:element name="learning_rate" type="xs:double" minOccurs="1" maxOccurs="1"/>
                          <!-- the alpha or the 'movement' is used in the back propagation formula to calculate new weights-->
                          <xs:element name="alpha" type="xs:double" minOccurs="1" maxOccurs="1"/>
                          <!-- the local gradient is used in back propagation-->
                          <xs:element name="local_gradient" type="xs:double" minOccurs="1" maxOccurs="1"/>
                          <!-- inputs as many as needed-->
                          <xs:element name="input">
                            <xs:complexType>
                              <xs:sequence>
                                <!-- Inputs optionally named in case order is necessary for definition -->
                                <xs:element name="input_name" type="xs:string" minOccurs="0" maxOccurs="1"/>
                                <!-- use appropriate type-->
                                <xs:element name="input_value_double" type="xs:double" minOccurs="0" maxOccurs="unbounded"/>
                                <!-- use appropriate type-->
                                <xs:element name="input_value_integer" type="xs:integer" minOccurs="0" maxOccurs="unbounded"/>
                                <!-- weight for this input-->
                                <xs:element name="input_value_weight" type="xs:double" minOccurs="1" maxOccurs="1"/>
                                <!-- added as a convenince for continuation of back propagation if the network is relocated, moved, cloned, etc-->
                                <xs:element name="input_value_previous_weight" type="xs:double" minOccurs="1" maxOccurs="1"/>
                              </xs:sequence>
                            </xs:complexType>
                          </xs:element>
                          <!-- end of input-->
                          <!-- bias start-->
                          <xs:element name="bias">
                            <xs:complexType>
                              <xs:sequence>
                                <xs:element name="bias_value" type="xs:double" minOccurs="1" maxOccurs="1"/>
                                <xs:element name="bias_value_weight" type="xs:double" minOccurs="1" maxOccurs="1"/>
                                <!-- added as a convenince for continuation of back propagation if the network is relocated, moved, cloned, etc-->
                                <xs:element name="bias_value_previous_weight" type="xs:double" minOccurs="1" maxOccurs="1"/>
                              </xs:sequence>
                            </xs:complexType>
                          </xs:element>
                          <!-- end of bias-->
                          <xs:element name="output">
                            <xs:complexType>
                              <xs:sequence>
                                <!-- outputs optionally named in case order is necessary for definition -->
                                <xs:element name="output_name" type="xs:string" minOccurs="0" maxOccurs="1"/>
                                <xs:element name="output_value_double" type="xs:double" minOccurs="0" maxOccurs="unbounded"/>
                                <!-- hypothetical value is a description of what it means if the neuron activates and fires as output if this is the last layer-->
                                <xs:element name="hypothetical_value" type="xs:string" minOccurs="0" maxOccurs="unbounded"/>
                              </xs:sequence>
                            </xs:complexType>
                          </xs:element>
                          <!-- end of output-->
                        </xs:sequence>
                      </xs:complexType>
                    </xs:element>
                    <!-- end of neuron-->
                  </xs:sequence>
                </xs:complexType>
              </xs:element>
              <!-- end of layer-->
            </xs:sequence>
          </xs:complexType>
        </xs:element>
        <!-- end of layers-->
      </xs:sequence>
    </xs:complexType>
  </xs:element>
  <!-- network-->
</xs:schema>

I hope this helps someone. This is open source. Please use it and pass it on if you find it useful.

Smart Road JSON or XML Template For Messaging and Data Transmission and Receiving

With the Internet of Everything here among us, and GPS is ubiquitous in vehicles, and cars will have full internet capability, it will not be long before we have Smart Roads.  Each road will have an IP address.

I began wondering what the data package would look like for Smart Roads, and then I was struck with the axiom that the best way to predict the future is to invent it.

So without further ado, I cracked together and XML file of what the Smart Road dataset would look like.  Here is my first crack at creating a Smart Road Data Standard:

<?xml version="1.0" encoding="UTF-8"?>
<!-- Smart Road Markup Language version 1.0-->

<smart_road>  
<!-- Main Element.  It consists of a header, data and trailer elements. -->


    <header>
      <!-- Header Element.  -->

      <ref_number></ref_number> <!-- This could be a database Primary Key Identifier.  -->
      <country></country>
      <state></state>
      <province></province>
      <county></county>
      <township></township>
      <ip6_address></ip6_address> <!-- Each Smart Road will have their own IP address.  -->
      <start_gps></start_gps>
      <classification></classification> <!-- There could be subclassifications like carriageways etc.  -->
      <end_gps></end_gps>
      <length_kms></length_kms>
      <maximum_speed_limit></maximum_speed_limit>
      <options></options> <!-- This could be a control element for suppressing some header info on subsequent exchanges.  -->

     </header>

  <data> <!-- Message Section.  -->
  <data_sent>
     <general_data> <!-- This section contains general data sent to the vehicles  -->
            <alerts>
                   <current_alerts></current_alerts>
                   <upcoming_alerts></upcoming_alerts>
             </alerts>
            <flags></flags>
           <messages>
              <alerts></alerts>
              <construction></construction>
              <law_enforcement></law_enforcement>
              <traffic></traffic>
              <weather></weather>
             <misc></misc>
            <user_specific>
                 <destination_address></destination_address>
                 <ack_flag> </ack_flag>
                <message_payload></message_payload>
                <delivery_receipt></delivery_receipt>
            </user_specific>
         </messages>
    </general_data> 
      <location_specific_data><!-- This section contains location-specific sent to the vehicles  -->
          <current_gps_marker>
               <current_maximum_allowable_speed></current_maximum_allowable_speed>
                <alerts>
                      <current_alerts></current_alerts>
                      <upcoming_alerts></upcoming_alerts>
                </alerts>
                <flags></flags>
               <messages>
               <alerts></alerts>
               <construction></construction>
               <law_enforcement></law_enforcement>
               <traffic></traffic>
               <weather></weather>
              <misc></misc>
               <user_specific><!-- It is anticipated that if a vehicle's onboard messaging is not working, one can billboard messages -->
               <!-- This can also be used to send violation notices and service-related messages to the vehicle -->
                     <destination_address></destination_address>
                     <ack_flag> </ack_flag>
                    <message_payload></message_payload>
                    <delivery_receipt></delivery_receipt>
               </user_specific>
              </messages>
       </current_gps_marker>
      </location_specific_data>
   </data_sent>
   <data_received>
        <vehicle>
           <type></type>
           <description></description>
           <direction></direction>
           <velocity></velocity>
           <timestamp></timestamp>
        </vehicle>
   </data_received>
  </data>

  <meta-data>
  <!-- Big Data meta data on usage stats etc.-->
  </meta-data>
  <trailer>
    <protocols_supported> </protocols_supported><!-- Various devices will have their own native Smart Road protocols -->
    <device_types></device_types>
    <end></end>
    
  </trailer>
</smart_road>

This is just a first iteration.  It needs to be tested and validated in real time.  It is anticipated that the vehicle will send its GPS coordinates to the database defined by the IP address, and the location specific data will be returned.

A Typical Privacy Policy

Here is a typical privacy policy.

Privacy Policy

What This Privacy Policy Covers

 This policy covers how Mythical MegaCorp and its subsidiaries (“Mythical MegaCorp”) treats personal information that Mythical MegaCorp collects and receives, including information related to your past use of Mythical MegaCorp services. For purposes of this Privacy Policy, “personal information” and “personally identifiable information” shall mean any information that can identify an individual directly or through other reasonably available means, except your name, business address, business telephone number, business email address and business fax number. By providing personal information to us, you signify your consent to Mythical MegaCorp’s collection, use and disclosure of your personal information in accordance with this Privacy Policy.

 This policy does not apply to the practices of companies that Mythical MegaCorp does not own or control or to people that Mythical MegaCorp does not employ or manage.

Information Collection and Use

General

 Mythical MegaCorp collects personal information when you register with Mythical MegaCorp and when you use Mythical MegaCorp services.

 When you register with Mythical MegaCorp we may ask for information such as your name, address, email address, phone number, credit card information, business license information, driver’s license information, postal code (zip code), and other financial information. Once you register with Mythical MegaCorp and log in to use our services, you are not anonymous to us.

 Mythical MegaCorp collects information about your transactions with us, including information regarding items you bid on.

 Mythical MegaCorp automatically receives and records information on our server logs regarding your sessions on Mythical MegaCorp websites and those of its subsidiaries, and related cookie information.

 Mythical MegaCorp uses information for the following general purposes: to contact you, to facilitate bidding, buying, and selling of items by you through Mythical MegaCorp and its affiliated companies, and to comply with laws and regulations related to purchases and sales of items by you through Mythical MegaCorp.

Information Sharing and Disclosure

 Mythical MegaCorp does not rent, sell, or share personal information about you with other people or nonaffiliated companies except to (i) provide products or services you've requested, (ii) when we have your permission, or (iii) under the following circumstances:

 We provide the information to trusted partners who work on behalf of or with Mythical MegaCorp under confidentiality agreements. These companies may use your personal information to help Mythical MegaCorp communicate with you about offers from Mythical MegaCorp. However, these companies do not have any independent right to share this information.

 We provide the information in response to subpoenas, court orders, or legal process, or to establish or exercise our legal rights or defend against legal claims, or upon receipt of a request from law enforcement;

 Mythical MegaCorp does not otherwise share personal information about website users with third parties unless doing so is appropriate to carry out a user’s request or it reasonably believes that doing so is legally required or is in Mythical MegaCorp’s interest to protect its property or other legal rights or the rights or property of others;

 We may transfer information about you if Mythical MegaCorp is acquired by or merged with another company to the extent permitted by applicable law. In this event, Mythical MegaCorp will notify you before information about you is transferred and becomes subject to a different privacy policy.

 We transfer information about you to sellers of items purchased by you, to buyers of items sold by you and to their service providers that they may retain to facilitate the assignment of items to Mythical MegaCorp sales venues, and to state and federal regulatory agencies as part of the title or ownership transfer process.

Cookies

 Mythical MegaCorp may keep track of the pages visited by its users by placing a small entry in text file, called a cookie, on a user’s hard drive. Cookies are text files we may place on your computer to understand user traffic patterns, technology used, usage data and aggregate demographics. They are also used to serve ads and authenticate you on the system. Cookies do not contain any personal information, but they do allow us to personalize the Service. You can remove or block cookies using the settings in your browser although doing so may interfere with your use of some of our site and Service.

Communications

 We reserve the right to send you certain communications relating to Mythical MegaCorp services and your account with Mythical MegaCorp, including but not limited to notifications, service announcements, and administrative messages without offering you the opportunity to opt-out of receiving them. Should you choose not to receive certain communications your access to or use of certain services may not be possible.

Links and Features Offered in Conjunction with other Providers

 To make our website more valuable to our users, we may feature some products and services that come from other providers through arrangements with companies that specialize in providing such services. We may share with these third parties such information as is necessary for them to provide the products or services. Our site may include links or provide access to third party sites, services and products. We do not

control the privacy policies and practices of third parties, and you are subject to the privacy policies of those third parties where applicable. If you can’t find the Privacy Policy of any third party via a link either from the site’s homepage or from the pages on which the products or services are offered, you should contact the third party directly for more information.

Data Storage

 Mythical MegaCorp may store your account information active in our databases indefinitely following the termination of your account with Mythical MegaCorp to the maximum extent permitted by law.

Confidentiality and Security

 We limit access to personal information about you to employees who we believe reasonably need to come into contact with that information to provide products or services to you or in order to do their jobs.

 We have physical, electronic, and procedural safeguards that comply with federal regulations to protect personal information about you.

 If Mythical MegaCorp learns of a security systems breach we may attempt to notify you electronically so that you can take appropriate protective steps. By using this website or providing personal information to us you agree that we can communicate with you electronically regarding security, privacy and administrative issues relating to your use of this website. Mythical MegaCorp may post a notice on our website if a security breach occurs. Mythical MegaCorp may also send an email to you at the email address you have provided to us in these circumstances. Depending on where you live, you may have a legal right to receive notice of a security breach in writing. To receive free written notice of a security breach (or to withdraw your consent from receiving electronic notice) you should notify us at privacy@remarketapp.com.

Changes to this Privacy Policy

 Your use of the our site and Service, and any disputes arising from it, is subject to this Privacy Policy as well as our Terms of Use and all of its dispute resolution provisions including limitation on damages and choice of law. We reserve the right to change our Privacy Policy at any time. We will provide a prominent notice on our website informing you of any changes in our Privacy Policy. The amended Privacy Policy will be effective immediately upon posting on our site, and your continued access or use of the Service following the posting of any such amendment will constitute full acceptance of the Privacy Policy as amended. We encourage you to refer to this policy on an ongoing basis so that you understand our current Privacy Policy. Unless stated otherwise, our current Privacy Policy applies to all information that we have about you and your account. You may determine when this policy was last updated by referring to the modification date found at the bottom of this Privacy Policy.