Automatic and Programmatic Two Phase Commit

Sybase:  An Organization with a Good Plan that Went Wrong.

The organization that I will consider here is Sybase, a still current relational database vendor.  At the time back in 1990, I was working for one of Sybase's main competitors, Ingres Software.  We built and marketed a relational database system as well.  In the 1990's distributed systems (client server) computing were all of the rage.  This extended to the database realm as well.  I was, at the time, the Manager of Distributed Ingres, which involved managing three product lines:  Ingres-Net, which allowed database clients to communicate with database servers over a host of networks, Ingres-Gateways, which allowed Ingres applications to store and access information in non-Ingres databases (these are database gateways not communications gateways), and Ingres-Star, our distributed databases management system, capable of both distributed queries (with a sophisticated performance based distributed query optimizer) and now distributed transactions.

In the realm of distributed transactions, back in 1987, Sybase had developed a two-phase commit capability with their database.  Two phase commit is the protocol that is required to execute distributed transactions (updating databases in multiple geographic locations at the same time).  Sybase had an excellent plan.  They could out do any competitor in the market with a two phase commit capability that only they had allowing Sybase database management systems to do distributed transactions.  Oracle didn't have two phase commit at the time, and neither did Ingres or Informix (among other things UNIX based database vendors, which IBM was not at the time).  I.e., Sybase was the first vendor in the open systems market to be able to do distributed transactions and held this lead for three years.

Along came our team at Ingres to change all of this.  We developed a two phase commit capability that surpassed the Sybase capability because it was transparent.  I.e., a distributed transaction ran just the same as a regular transaction, the database user didn't even need to know that the databases being updated were in different locations.  I positioned this as "automatic two phase commit," which killed interest in the Sybase product, which I depositioned as "programmatic two phase commit."  The problem that Sybase had is in order to use two phase commit with their product you had to explicitly program the two phase commit protocol.  It was not transparent.  When we announced our automatic two phase commit product, it was so well received in the market that we got front page press in almost all of the important trade magazines and newspapers.  We had executed a coup that severely hurt Sybase.  That was my job, I was the Manager of Distributed Ingres.

So in this case, an innovative competitor came into the market, Ingres, and introduced new technology that made Sybase's product obsolete, and a technology that also changed the database market.  Sybase's plan became obsolete.  Customers were now asking for automatic two phase commit.

Impact of the Socio-Technical Plan for Two Phase Commit

In terms of the sociotechnical plan, as far as  people's interaction with the technology, ours was much easier to use.  With Sybase's product you had to be a programmer.   With our product you didn't have to be technical at all, just say update (and the distributed update is done automatically).  The sociotechnical plan was to make two phase commit easy to use and to require no programming capability.  The distributed system automatically did the work for you.

Relevance:  The Need to Two Phase Commit and Distributed Transactions Today

This technology is relevant and still relevant as the world is distributed.  Computers everywhere are connected on networks and need to work together to get tasks done (distributed computing).  This is the case with the internet and any installation of computers and networks in any organization.  Companies continue to need to be able to do transactions in a networked environment.  I.e., they need distributed transaction capability.

Three Forces That Could Affect Automatic Two Phase Commit

At the time, there were several forces that Ingres needed to consider upon release of the automatic two phase commit capability.  One, other vendors could follow on and develop their own automatic two phase commit capability.  Oracle, several years later, with Oracle 7 did just that.  Sybase eventually, several years later,  developed automatic two phase commit as well.  Two, another possible force would be standards that allowed heterogeneous distributed transaction capability.  At the time that Ingres released automatic two phase commit it only worked on Ingres databases.  I.e., what we had was homogeneous two phase commit.  Needed also by the market was the ability to update multiple databases of different types in different locations.  This standard never came about, and heterogeneous transactions only exist in a small measure by database companies that have modified a database gateway or two to do a restricted type of heterogeneous transaction.  The market went a different route, to the internet and to cloud computing and big data.  Big data is really about reads.  You read massive amounts of information and then analyze that information for patterns.  It is not very update oriented.  A third possible force would be transaction managers that handle distributed transactions like IBM's CICS.

References

These are all books from my bookshelf purchased when I was in the database industry.  Date , from IBM, wrote the classic text on relational database systems.

Bal, H.E. (1990).  Programming Distributed Systems, First Edition.  Silicon Press. Summit, New Jersey..

Ceri, S., Pelagatti, G. (1984).  Distributed Databases:  Principles and Systems.  McGraw-Hill,  New York, New York.

Date, C. J. (1983).  An Introduction to Database Systems:  Volume II.  Addison-Wesley,  Reading, Massachusetts.

Date, C. J. (1986). An Introduction to Database Systems:  Volume I, Fourth Edition.  Addison-Wesley, Reading, Massachusetts.

Khanna, R. (1994).  Distributed Computing:  Implementation and Management Strategies.  Prentice Hall, Edgewood Cliffs, New Jersey.

Serendipity, Error, and Exaptation

Innovations can come about by careful planning and experimentation, but also can come about accidentally by serendipity, error, and exaptation.  These terms will be explained below.

An Invention that Came About by Serendipity

Microwave Ovens were discovered serendipitously.  In 1945, Percy Spencer, a self-taught engineer that worked for Raytheon, was working on microwave based communications systems (similar to radar, but microwave based communication systems are based upon a higher frequency electromagnetic wave than are radio wave based radar systems).  When working with one of these microwave devices Percy noticed that his pants were getting hot and a chocolate candy bar in his pocket melted.  The microwave communications system was clearly emanating energy as heat.  Percy realized immediately that this could be used to heat food quickly, and Raytheon immediately got a patent for a microwave oven.  However, it wasn't until 1967 that microwave ovens were small enough to go into peoples' kitchens.  This is when the microwave oven craze took off.

An Invention that Came About by Error

Micro-electromechanical Systems or MEMS.  MEMS, which are small (1 cubic millimeter) silicon chips that can function as sensors.  They were discovered by a lab error.  Jamie Link was a graduate student at U.C Berkeley in Engineering working on silicon chips.  In one of her experiments she broke the chip she was working on into many small fragments, i.e., by a lab error.  However, she discovered that some of these fragments were able to function as sensors.  Further work was done by DARPA (the Defense Department) to develop these small chip fragments into what they named Micro-electromechanical Systems or MEMS.  MEMS can detect physical properties like temperature, light, chemicals, magnetism, or vibrations and convert the information into electronic signals that can be uploaded to a computer for data analysis.  

An Invention That Came About by Exaptation

An example of exaptation may be the change in use or repurposing of metal detectors.  The metal detector was invented in 1881 by Alexander Graham Bell.  Metal detectors were originally used in the 60's in looking for metals underground in mining operations, and also in wartime (Vietnam and other wars) to locate buried landmines.  A similar use was by individual looking for precious items underground such as rings or jewelry.  An exaptation or reuse of the metal detector is the modern use at airports and other places of events to insure that no-one entering the airport or venue is carrying a gun or metal weapon. This clearly was a repurposing of an invention, i.e., it was an exaptation.

Meaning of the Terms

Serendipity

Serendipity refers to an accidental discovery (Wade, 2012).  Many great innovations have been discovered accidentally, i.e., serendipitously.  One of the most famous serendipitous discoveries is penicillin.  

Error

Discoveries by error are cases where something that wouldn't have been discovered in all probably by a normal process was discovered to some error (Davila and Epstein, 2014).  Examples are experiments that go wrong and someone accidentally discovers something.

Exaptation 

Exaptation is repurposing of an invention.   I.e., something is invented to do one thing and later that invention is used to do something else not originally conceived of.  The original terms comes from evolutionary biology where a trait is readapted for a different use than was originally intended.  An example is some dinosaurs had features (for warmth) but later some of these species evolved into birds that used the feathers for flying.  However, as concerns technological exaptation,  this would mean finding a new use for a technology which was not part of its original intended use.  

References

Wade, W. (2012).  Scenario Planning:  A Field Guide to the Future.  John Wiley and Sons, Inc.,   Hoboken, New Jersey.

Davila, T., Epstein, M. (2014).  The Innovation Paradox.  Berrett-Koehler Publishers, Inc., San     Francisco, California.

How Apple Disrupted the Music Business

Disruption to the Music Business Model

            The industry that will be covered is the music industry and the tremendous disruption that the iPod and iTunes and digital music did to the traditional music industry.  This paper is based on several articles, including a case study entitled "The emerging music business model: back to the future? (Koster, 2011)."

            While artists, studios, and retail music outlets were busy doing traditional forecasting of the demand for their labels, Apple was stirring up the industry with not only a new technology for music (digital downloads) but an entirely new business model for the music industry.  This eventuality of listening devices like the iPod, and digital distribution of music through websites was completely unforeseen by the music industry and forever changed the music industry landscape.  No one in the music business had done scenario planning to anticipate and respond to the development of digitally downloaded music.

            The Apple iPod listening device was introduced in October of 2001.  The Apple iTunes music download business (first downloaded to MacIntoshes) was introduced in February of the same year.  The music industry mistakenly saw this move as a new device on which to listen to music.  They did not understand that it would completely redefine the music business (Johnson, Christensen, and Kagermann, 2008).

The Traditional Music Business

            Let's consider the traditional music model.  Before modern recording technology, music was only available at live concerts.  Concert halls existed to showcase symphonies and other types of music, as theaters also showcased musicals and opera houses, opera.  A fundamental change to the music business came about when Thomas Edison invented the first sound recording and playback device, the phonograph in 1877.  After that point,  a record industry began to develop.  Around that time an Italian inventor, Guglielmo Marconi, invented the radio in 1895.  The first public radio broadcast was in 1906.  The radio was especially popular before the invention of television, and regularly included music broadcasts as well as dramas, news, and public announcements.  The radio continued to be popular for music broadcasts which was augmented in the 60's due to transistor radios (invented in 1954), as the first portable music listening device.  The transistor radio may be considered the iPod of the 60's.  

            Given these technologies, the traditional music business may be considered by the model below.

Figure 1:  The Traditional Music Industry Business Model (My Diagram).

            Artists have several venues by which to distribute their music to the public.  One is by traditional concerts.  After television had been invented, many artists appeared on television to promote their concerts and album sales.   An example is the famous Ed Sullivan show which promoted many artists including the Beatles on their first American tour.  Later, music specific television stations appeared, such as MTV (Music TV).  At that point, many artists also released music videos.  Musicians also sold cassettes, record albums, and CDs (depending on the time period), which spurred the phonograph, cassette player, and CD player industries, as well as car technology for music listening.  Artists would typically record their songs at a recording studio, and bundle the songs into an album unless releasing a single, and then cut the record, cassette, or CD.  This was then distributed through music retailers, where consumers bought albums or singles and played them on their respective listening devices.  Albums and singles were also played on the radio stations and consumed by the public on radios, which was primarily used by artists to promote their work and spur album sales.

The Digital Music Industry

The digital music revolution changed the traditional music industry model dramatically. 

Figure 2.  The Digital Music Industry Business Model (My Diagram).

            The digital music business is somewhat different than the traditional music business.  Artists still record their music at a recording studio, but the output is a digital music file in some popular format, such as MP3, which is then distributed through music websites.  On these websites, notably iTunes, music selections can be purchased and downloaded individually.  Albums do exist, but most users now consume individual songs.  This gives users more control over what they want to listen to, and they can, on their digital devices, create playlists with just the songs they want to listen to in the order in which they want to listen to them.  This was not possible in the traditional music industry model.  I.e., consumers have more control over the music they listen to.  A typical single song on iTunes sells for between $.99 and $1.29, i.e., a low price.  Full albums are somewhat more expensive.  Consumers also still listen to music on the radio, which is used primarily as a promotional device for artists as is television.  By hearing these songs or seeing the performers, consumers may decide to purchase a ticket to one of their concerts or to pay for a digital download of their songs.  Digital videos may be downloaded and watched on a digital device and cost more.

            A big and disruptive problem for artists in the digital music age is getting paid for their songs (Kwok, 2002).  Songs are cheap, and often downloaded illegally without customers paying anything.  There have been many attempts by various bodies and artists to crack down on illegal downloads, but the problems persist, and this means loss of income for artists for their work (Swatman,  Krueger, and Van Der Beek, 2006).  To compensate, many artists have beefed up their touring, i.e., their concert and live event venues for which they can receive ticket sales at a handsome price.  Artists still receive a certain amount of income from digital downloads,  and older artists receive royalties from their earlier works offered in digital form, such as the Beatles.

Questions and Answers

How Does Scenario-Type Planning Support Planning and Innovation for Change?

            Scenario planning supports planning and innovation for change by using multiple alternative scenarios to prepare for eventualities that may or will change the market, the demand for technology, or the technology itself.  If one is planning an innovation, the opportunity can be lost due to unforeseen developments:  competitive technology, changes in customer demand, market developments, industry developments, emerging regulation and standards.  If one doesn't create various scenarios to react to these developments, the company can be caught off guard and lose an opportunity for innovation.  Consider Kodak, that was the preeminent company in the analog film business.  They did no scenario planning and did not anticipate the development of digital cameras and the digital camera market.  Now they are bankrupt.

What Forces are Involved, and What Impacts Do They Make?

            In the case of Apple disrupting the traditional music industry and creating a new digital industry, there were several forces involved.  Musicians are not typically technical nor are others in the music industry.  They are hence not in a position to foresee technological developments nor to think about how those developments may affect them.  This was clearly the case in the iPod and iTunes disruption.  The forces were the development of a new listening technology, the iPod.  Another force was the development of the internet as a method by which songs could be distributed.  I believe no one in the traditional industry saw that coming or understood that.  Steve Jobs of Apple saw that coming and understood it and hence started and stayed ahead of a new way to consume music.  Another force was that music could be consumed on multiple devices:  computers, iPads, smart phones, digital music enabled cars, not just one type of dedicated device, like a CD player.  Another force was consumer choice.  With digital music, consumers could download just the songs they wanted to listen to, and with playlists, put the songs in the order in which they wanted to listen to them.  This was not possible with the traditional record, CD, or cassette technology.  Another force was consumer convenience.  Any song could be downloaded from the internet on any digital music device.  It wasn't necessary to go to a music retailer outlet to get one's music.  Another force of consumer choice was the variety and extent of the songs available on the internet, literally millions of songs.  Retailers may be limited in the number of CDs or albums they carried.  Again, a force for consumer convenience is that many personal functions could be made available on one portable device, a smart phone, which allowed listening to music as well as taking pictures, texting, making phone calls, sending emails, browsing and using the Internet, and executing thousands of apps.  I.e., there was no need for a separate dedicated listening device as with the old technology.

            Although the digital music industry in some ways hurt artists (their income), it was a big win for consumers of music.

Intended Use of Scenario Planning

How Will You Use Scenario Planning for Future Innovation Efforts?

            I have never done scenario planning per se, but have done more than just forecasting.  Of course in Engineering and particularly in Product Marketing, I have forecasted the number of units and tracked which products are selling and which are not, i.e., which are to be continued and enriched and which are to be discontinued.  Moreover, as an Engineering Manager and Product Marketing executive, I have had to outguess where the market and technology is going, where the competitors are taking their products, and how the industry will develop.  But I never had a multiple-point plan that planned multiple alternative futures and our reaction to those possible futures.  I will use scenario planning in the future.  It is useful both for business planning and technology planning purposes.

Does the Scenario Plan Account for the Social Impact of Change?

            The scenario plan that will be used will account for any foreseeable change in the market and  consumer behavior.  An innovation has social impact.   It changes the way we use something, do something, or think about something.  In general, in a scenario plan, one wants to consider the driving factors affecting the technology or technology market (which could include changes in consumer behavior).  Any innovation will have a social impact but the extent of the splash it makes depends on how innovative it is when it is released to the market.  Is it unique?  Is it something the competition doesn't have?  Will the competition have something like it by the time it is released?  Does it lead the industry?  Is it a technology that is still relevant?  These are all questions that need to be asked in scenario planning to gauge the social impact of the change the innovation brings about upon release to the public.

Conclusion

            When Apple created the iPod and iTunes, people in the music industry were not alarmed.  This is because they completely did not understand the grandeur of what Steve Jobs was creating.  He was not just creating a new music listening device.  He was not just creating a music download service to provide songs for that device.  He was completely disrupting and changing forever the music business with a new business model:  the digital music revolution (Kusek and Leonhard, 2005).  Artists, recording studios, managers in the music business, did not understand Job's vision and did not do any scenario planning to plan for such a future, disruption, or change to  the music business.  They were completely caught off guard and had to change the whole way they produced and distributed music to play in the new market.  They didn't do scenario planning.

            As an aside, I will comment that changing the music business is not the only revolution the genius Steve Jobs forged.  He changed the computer business by creating the first personal computer (with Steve Wozniak the technical wizard).  IBM and other companies soon followed suit, having to play Steve Job's game.  He revolutionized the world with at least five revolutions:  the PC revolution, the digital music revolution, the smart phone revolution, the tablet revolution, and even the Apps revolution.  Geniuses like Einstein come along infrequently and hit the genius mark over and over and over.  Jobs was not a scientist or even an Engineer;  he did not program or build circuits, although he had a deep understanding of technology.  I consider Jobs one of those exceptional people who hit the genius mark over and over as an entrepreneurial genius.

References

Johnson, M. W., Christensen, C. M., & Kagermann, H. (2008). Reinventing your business model.             Harvard business review, 86(12), 57-68.

Koster, A. (2011). The emerging music business model: back to the future?. Journal of Business   Case Studies (JBCS), 4(10), 17-22.

Kusek, D., & Leonhard, G. (2005). The future of music: Manifesto for the digital music      revolution (p. xi193). S. G. Lindsay (Ed.). Boston: Berklee Press.

Kwok, S. H. (2002). Digital rights management for the online music business. ACM Sigecom        exchanges, 3(3), 17-24.

Swatman, P. M., Krueger, C., & Van Der Beek, K. (2006). The changing digital content   landscape: An evaluation of e-business model development in European online news and       music. Internet Research, 16(1), 53-80.