Personal Assistant Technology

From New Media Business Blog

Jump to: navigation, search
Apple's Siri assisting a user who is lost with GPS

A Virtual Personal Assistant or VPA is a type of computer software that assists humans with daily tasks such as adding appointments to a calendar or answering trivia questions (not to be confused with a Virtual Assistant, an individual remotely providing professional assistance to clients). Modern forms of VPA's are run from mobile phones or computers. Currently, the most common referenced virtual personal assistant is Apple's Siri. A VPA can accept commands through voice or text commands from humans. Roles performed by a VPA are traditionally performed by a human manually on their stationary or mobile computing device.

A VPA is an automated digital system that can interact, understand and perform tasks and requests from the user. The program usually uses the Internet gather, process, and compile relevant information for answering questions or performing tasks posed to their systems. In the past, businesses would hire people to perform these types of jobs:

  • take notes
  • make copies
  • give reminders
  • send/receive information and relay it to the boss
  • retrieve snacks/beverages


Functionality and Purpose

Stivers, Personal Assistants.
Humans had to assist with even the tiniest of tasks.

VPAs are coded using logic algorithms into software applications that are then installed onto hardware. The hardware requires strong processing power and enough memory to run the software smoothly. At the moment, VPAs can receive input using speech recognition software and text input.

When the user asks the VPA a question, the software, usually connected to the Internet, will browse internal and external databases to gather relevant information. Although this is very useful, realistically the system is limited to its ability to access information Filter Bubble. Information retrieved is not always one-hundred percent accurate as online search engines tend to be biased on their results.

The latest iPhone from Apple uses Siri, a personal assistant software that uses natural speech recognition. This type of voice recognition software uses words from the user's commands to try and create connections to other related words to build the context of your questions and answer them appropriately.

The purpose of using a personal assistant, whether virtual or not, is to help to organize and simplify day-to-day tasks. Some useful functions that you can do with current VPAs:

  • Set up reminders
  • Send emails or text messages
  • Make phone calls
  • Open applications
  • Retrieve useful information such as the weather, time, or directions
  • Answer questions using internet search engines
  • Provide recommendations for services based on location


Shakey the Robot at Stanford University in 1972

The history of VPAs is closely related to the development of Artificial Intelligence (AI) technologies over time. Even before the 1900's, philosophers, such as Gottfried Leibniz, envisioned a world that at some point would have artificial intelligence beings would be smarter than humans. This was based on the assumption that human thought and reasoning could be mechanized.

In the 1940's and 1950's, scientists, mathematicians, psychologists, and engineers began to invest heavily in exploring the possibility of creating an artificial human brain. This lead to the idea of Game Artificial Intelligence, which is a computer machine that has the ability to compete against a human in a game such as chess. In 1956, the Dartmouth Conferences were hosted by leading researchers in the newly created AI industry and senior architects by IBM. The goal of the conference was to begin brainstorming about AI technology.[1]

From 1956 to 1974, there was heavy investment in the industry, which enabled computers to be able to solve more complex mathematical problems and learning to speak English. Even though there was a lot of new discoveries and advancement, the industry faced numerous challenges that they could not overcome which lead to a roadblock in research. The challenges that caused these delays are described below:

Computer time required to compute complex problems 
Complex problems in unison with constrained computer power meant that some problems would require extensive amounts of time to solve problems
Commonsense Knowledge 
Computers at the time were unable to interpret all of the possible meanings of different words and phrases which meant they were limited in the types of questions they could answer
Human Reasoning 
Computers just did not have the same amount of intelligence as computers, which meant that they could not reason and use common sense like humans could
Limited computer power 
The computers that were currently available simply did not have enough RAM or memory to handle the processing that was required to solve the complex problems that AI was taking on

Modern AI involves expert systems and the push to solve the problems encountered in the 1970's. The first solution to these problems were Expert Systems which focused on solving problems within a defined subject area and range of knowledge. This way, less computing power would be needed and the system would be able to solve problems quicker. Intelligent computers have more recently found their way into the hands of consumers, as shown in the Examples of Virtual Personal Assistants section.

Features and Technology

Speech and Semantic Recognition

For Smart Personal Assistant Technology like Siri and Watson, the greatest advancement from personal digital assistant devices is that they have speech recognition and near natural language processing capability. [2]

Although speech recognition is not fine-tuned to recognize at 100% accuracy, it is the function of the future. Much like Web 3.0, this is machine-human interaction 3.0. It introduces the interaction between human and machine using natural language. According to BBC, the future of interaction with our devices is voice control. Imagine talking to your car, phone, computer, and other devices that assist in daily tasks and activities that are able to respond and take your commend like a human personal assistant. [3]

However, the ability to respond to queries for VPAs is just a series of machine manipulated algorithms, it does not understand the meaning. The cognitive ability is absent. According to a study, experiments on Siri show only 83% accuracy in interpreting meanings.[4] Nevertheless, computer programs have their shortcomings when it comes to recognizing semantic ambiguity and emotions.

When one enters an information query into Google search, a query is built and the job will be assumed by the search engine to complete the search and return the user with information related to the query. This function is called keyword search, which is not a usual way of verbal communication in daily life. Therefore, speech controlled search systems are the future and have the capacity to satisfy the needs of consumers. Nevertheless, the semantic recognition of speech control search engines is still not as developed as keyword search. According to Joe Recomendes, a digital marketing strategist, “Keywords are great now, but natural language may be equally important soon". [5]

Personalized and On-Demand Search Functionality

In the coming years, machines that are capable of learning our preferences can be programmed by any person because tailoring our preferences is as easy as using your assistant to search up a dinner or travel plan. Personal assistants can learn from our search history, so in a way we do not need to learn a mark-up language to interact with our machines anymore, they are capable of our language. With their memory and analytical ability, they are more capable of knowing us and providing us with the personal experience than any other human persona assistant. Mahoney said in his article, "As more machine learning capability gets implemented and more of these systems sort of build themselves, you'll see better and better coverage. The systems will learn from use about what kinds of things they need to cover, and they'll get smarter over time." [6] There is of course dark side of this technology, which raised the awareness of people who want to keep their personal information or preferences as private and safe as possible. It raised not only a security question, but it shakes some fundamental ground of our Internet world, and how we live our “lives” in it. Question has been asked whether the Internet is indeed free, in a way that we can exercise our free will and expand our knowledge.

The Filter Bubble

The Filter Bubble

The filter bubble is a term first coined by Eli Pariser. He suggested in a TED talk that Internet users live in a world that is too tailored to what they are interested in over time; that they cannot jump out of the circle we have created for themselves. Modern search engines collect user information which can be used in the future to tailor search results to topics that match search histories. This can work both ways, it provides great convenience for users because it shortens their path to what they want. On the other hand, it can also limit knowledge.

Most people consider web search as a way to expand their knowledge, but it also limits their knowledge to a small circle. Therefore, it also becomes one of the limitations for systems like Siri. These speech recognition tools have the “learning” ability to collect user information and start tailoring queries. Over time, it can provide users with more personalized assistance. Nevertheless, this will run into the black hole of “filter bubble”. Search engine providers will serve as the gate keeper for what information users are receiving.

In the economy where information can be monetized, this practice is concerning for most users because knowing preferences and being close to the source of spending makes consumers extremely vulnerable to advertising. Search providers may easily present users information of their choice rather than the best possible choice for the user.

Security Concerns

One of the key features of VPAs are their connectivity. They allow remote access of information from cloud servers, and by combining information from different sources, users can find an answer to their query. Therefore, the security concern, especially where Bring Your Own Device (BOYD) is allowed, is a major concern of companies. Siri is banned from IBM because of it’s database storage is in a remote location.[7]

Other security concerns like unauthorized access to personal devices that contain sensitive information for individual users is large. Biometrics with static facial recognition can be breached using a photo, and Siri’s voice recognition cannot distinguish the owners’ voice with an unauthorized user's. If a stranger inquires about the owners’ information, confidential information that is “known” by Siri will be disclosed. Therefore, human override is still necessary to protect information. Personal assistant programs like Siri can be taken advantage of if we do not have a more secure access mechanism. In the video interview about Siri’s business model, they are planning to store credit card information to make purchasing small impulse items easier. But as far as security is concerned, it could put the owner at risk of identity theft. [8]


Although personal assistant technology is still a relatively new technology, it has recently hit the consumer and business markets. The most prominent examples of mainstream personal assistant technologies are Apple's Siri and IBM's Watson.

Apple's Siri

A marketing video released by Apple demonstrating some of Siri's features

Siri is defined as an “intelligent personal assistant”. It is intelligent because it has speech recognition capability, so it takes commands in natural language. One can talk to Siri and search locations, businesses, send messages, book meetings, and set reminders.[1]

Siri was first developed by Siri, Inc. established in 2007 by Dag Kittlaus (CEO), Adam Cheyer (VP Engineering), and Tom Gruber (CTO/VP Design), and Norman Winarsky from SRI International's venture group. Siri began as an Apple App Store application and later was acquired by Apple on April 28, 2010. On October 4, 2011, Apple launched the new iPhone 4S which integrated Siri as a core operating system feature. This was presented as a big selling point for consumers.

Original Business Model

A description of Siri’s original business model

According to Tom Gruber, Siri was first developed with the goal of creating revenues by providing voice controlled search to users. Siri would identify the location of the user and where they usually spend their money. Siri would then sell advertising and acquire royalties by making listings for local businesses. The application would not have an initial purchase fee so the company could target the mass market and gain network effects.

Apple's New Business Model

Apple's former CEO in 2010, Steve Jobs, discussing how Apple plans to generate revenue from mobile search

After Siri was acquired by Apple in 2010, the business model change drastically. According to Apple, generating revenues through Siri is not a goal of the project; it is simply used as a feature that Apple markets to sell more devices.[1] The threat to Google search is immense considering the rivalry between Apple and Google, and Siri does not use Google search. Apple has partnership with many listings providers like Yelp, Open Table, and Rotten Tomatoes. Siri bases its facts search on Wolfram Alpha.

Information Bias and Limitations

There are many limitations to Siri at this stage in 2012. For example, geographical limitations, semantic recognition, and language limitations are among the most prevalent on the list. There is, however, another hidden set-back, which is related to the search engine’s information provider. As mentioned above, Google is not used in Siri’s search, which is major opportunity for Apple to cater to many people who are used to or prefer Google searches. However, the two competitors will not compromise easily. Nevertheless, if users intend to search Google, they need to give more specific instructions with their search to get the results they need.

An information bias is prevalent with Siri, as shown in this video. As discussed earlier, Siri has partnership with some listing services, and thus it is possible that the preferred listings will be shown instead of the closest business to the user. There are elements of advertising related to web search results, and user preferences and information can also be collected for commercial use.

IBM's Watson

Alex Trebek, Ken Jennings, IBM's Watson, and Brad Rutter competing on IBM's Jeopardy Challenge in 2011

Watson is a deep question and answer computer that was created by IBM to compete on the popular television show Jeopardy against human competitors. Watson performed exceptionally well, defeating both of the humans it was competing with on the show (source)

Watson, created and developed by IBM, was created over three years with the goal of understanding natural language and being able to compete against human players in the popular TV game show, Jeopardy. Although this was the initial goal, IBM is continuing investment into Watson in order to bring it to the commercial markets that involve extensive processing of natural language to solve problems, such as the industries of Healthcare and Finance. Watson's main goal is to transform the way that organizations operate by changing the way they think and act to achieve better outcomes. Since it was envisioned in 2004 by IBM, the system has been further developed and is on the edge of reaching the commercial market.

The first 11 minutes of Watson competing on Jeopardy!

How Watson Works

Watson goes through a series of four different steps in less than five seconds to generate answers and confidence levels. Once Watson receives the question in text form, it begins its process. Each of the four steps is described below:

  1. Question Analysis
    1. The question is broken down and parsed into different parts
    2. Watson determines what type of question is being asked and what is being asked for
  2. Hypothesis Generation
    1. Watson searches through hundreds of millions of documents to come up with a large list of possible answers
    2. In this step, it is more important to generate a lot of answers rather than narrow down to a few answers so Watson has more to choose from in the next steps
  3. Hypothesis and Evidence Scoring
    1. This is the most important step of the answer generation process
    2. During this step, Watson finds evidence to support and defend correct answers, while also finding evidence to downgrade wrong answers
    3. More data is collected on possible correct answers to decrease the number of possible answers
    4. Evidence is also rated in terms of reliability and how timely the information is to determine if it is quality evidence
    5. Several algorithms work in parallel to find this information
  4. Final Merging and Ranking
    1. Watson uses its experiences from answering similar questions in the past in this step to weigh and rank the different types of evidence gathered in step 3
    2. At the end of this step, Watson will estimate its confidence level of possible answers
    3. The answer with the highest level of confidence and most evidence gathered is ranked as Watson's top answer

Technical Details

Watson's Server Room

The unique thing about Watson is that it does not access the Internet to gather its information when answer questions. It does however, require a massive amount of computer hardware to answer questions as quickly as it does and to store vast amounts of information.

According to IBM, Watson is made up of a cluster of 90 IBM Power 750 servers. These servers use 3.5GHz POWER7 eight-core processors, which gives it a total of 2880 POWER7 cores. The cluster described previously has a total of 16 Terabytes of RAM and the system that contains all of Watson's data is on four Terabytes of disk. This power is demonstrated by Watson being able to process 80 trillion operations per second. This allows the system to answer questions as fast as it does on Jeopardy. The operating system that the massive computer uses is Power Linux.[1]

Commercial Applications

IBM is harnessing the power of Watson, as demonstrated on Jeopardy, by applying it to commercial applications. The first two areas that IBM is targeting are healthcare and finance. They hope to improve the work environment and achieve different goals in each industry. The role that Watson aims to play in these industries is to sift through vast amounts of data that is always being updated and revised and provide it to professionals who need it and don’t have time to find it themselves.

Advancing the Future of Personalized Health Care [2]

“Watson uses natural language capabilities, hypothesis generation, and evidence-based learning to support medical professionals as they make decisions.” [3]

Since the medical industry contains vast amounts of data that is buried within millions of sources, Watson is the perfect fit for doctors to query with natural language questions to find the data and information that is relevant to the case they are dealing with. When a physician is with a patient, they will be able to input all of the symptoms and information they have about their patient and Watson will break the information down into small pieces and run it through all of the articles and other information it has access to. It will then list recommendations of diagnoses and a score of confidence. With this information, the physician can then make a decision of what to do. This allows physicians to make more informed decisions for patients.

Memorial Sloan-Kettering Cancer Center (MSKCC) and Watson Collaboration to Help Oncologists One of the first pilot projects is IBM’s partnership with MSKCC. Watson will assist Oncologists (physicians specializing in cancer treatment) to quickly identify cancer histories and make evidence-based decisions. The video below describes the partnership.[4]


IBM is also looking to put Watson to work in the Finance sector. According to IBM, over 9000 pages of financial news is published each day, and it is virtual impossible for those who work in the industry to keep up with all of this news when they are making decisions[5]. The goal of Watson here is to assist these professionals in finding the relevant data that they need to make decisions.

Similarly to Jeopardy and Healthcare, Watson will be able to process natural language and make recommendations to a user along with its level of confidence on those options. The ability of Watson to take the context of the situation and information into consideration is what will make it very useful in the financial services industry.

On March 5, 2012, IBM entered into an exploratory agreement with Citigroup to explore possible opportunities with Watson in the industry[6]. As Citibank is looking to be the world's leading digitally enabled bank; the goals and functionality of Watson align perfectly with their vision. Evaluations will be conducted to discover the best ways that Watson can improve customer relationship management by bringing customers timely and relevant information as fast as possible. Although this partnership was made very recently, it is one of the first steps in putting Watson to work in the financial industry.

Automated Phone and Internet Attendants

Automated Virtual Assistant, "Lucy" used by O2 telecommunications

The voice recognition automated phone attendant is a relatively mature virtual assistant technology being utilized by businesses. It is commonly used in telecommunications and internet banking, but other industries have also found it useful in routing calls. In call centers, the automated virtual assistant assists customers in performing simple tasks such as retrieving information, checking account balances, or verifying business addresses.

This software brings speedy services to customers, especially in call centers that have a high volume of customer calls, as it decreases the number of calls going directly to live representatives. The biggest limitation of automated phone attendants is that they lack personal interaction and in many instances, frustrate customers who have negative experiences with automated assistants that do not provide them with what they want.[7]

Automated attendants are also starting to be utilized on websites. They appear to the user as an intelligent agent, but are very limited in the number and complexity of questions that they are capable of answering. One example is Lucy used by O2 telecommunications

Adoption in the Workplace

The Value Chain Model

In terms of business value creation, human decision making is still essential in R&D, marketing, and customer service. In the service industry, it is still essential to involve human interaction because computers cannot yet handle the complex decision making that humans face each and every day.

In order for traditional businesses to adapt VPAs to compliment or even replace human employees, some primary value creation activities shown in the diagram on the right, the technology needs to outperform humans in these tasks. Some companies have started to accept the virtual assistant in the form of cloud outsourcing. The outsourcing of administrative functions to more economic options overseas is a more viable virtual assistant. Furthermore, more companies are embracing the online self-service option for customers' convenience. It can also be a better option for service oriented companies to eliminate time constraints if online help is available. According to Gartner, "By 2015, 50 percent of online customer self-service search activities will be via a virtual assistant for at least 1,500 large enterprises."[8].

Workforce Restructuring

The cost effectiveness of work place business process automation will make businesses realize that some peripheral tasks can be accomplished by computers, or can be outsourced to virtual assistants. Therefore, the workforce, especially in the office, is starting to restructure itself towards IT support other than having administrative position. For example, companies use automatic attendants instead of front desk clerks.

Nevertheless, complex decision making process is still dependent on human intelligence. More software applications are used as decision support and analysis tools, but they cannot replace humans at this stage because there is the question of whether a machine can be programmed or "taught" to adapt moral values. In terms of business decisions, there needs to be a combination of information and flexible adjustment to any uncertainties or risks, so computers are good at computing data and reducing workloads so some alternatives can be generated. Ultimately, human decision making is still needed to incorporate moral values. For example, a decision that optimizes the expected value for the company is not necessarily the right decision in terms of social responsibility or public relations.

There have been real concerns for comparatively less skilled workers in many companies. According to an article written in The Atlantic, the “possibility of technological unemployment: The basic idea was that at some point, the equilibrium wages for workers might fall below the level needed for subsistence." [9] The restructuring of workforce skills puts pressure on individuals to attain higher education. Even a middle management job that requires certain skills is not guaranteed. Because a machine's decision making skills are far more sophisticated, so with software aided decision support system, managers might not be needed anymore in the near future.[10]

Business Intelligence

Info Engineering and Business Intelligence

In the age of information, the ability to interpret information and make it useful for decision making is very valuable. Information is readily available, so the ability to sort and interpret the business value is very important.[11]

PDA is essentially a search function that sorts, link, aggregates, and build queries to extract information to form of knowledge, which we can serve managers as decision support.[12] For example, In the form of personal assistance, it provides us with information on restaurant locations so we can pick one that has the best rating. It can also provide us with location information so we know where we are going. In terms of business intelligence assistance, the technology can do the repetitive work of sorting and retrieving information by query, and the work of calculating and analysis will also be assumed by software, and the resulting intelligence is fed back into the business to make better business decisions in the future.

Business Process Automation

Digital personal Assistants are used in many areas to assist business process automation and data storage. It makes traditional manual processes more efficient by using computer automation. Computers have much larger capacities than the human brain to store and search for information. For example, in a retail setting a sales clerk can use PDA to track inventory, which is more efficient than manually checking. [13]

Cost Effectiveness

From a business point of view, one of the biggest bottom lines is the economic bottom line. Profitability is always the concern for managers when they make decisions to hire extra employees or implement new technologies. For example, the NPV evaluation would tell one if return is worth the investment over the life span to invest in that option. However, since human resources are generally the biggest expense of a company [14], it is hard to measure the return and there are many risks involved.

Therefore, by adapting an automated business process, companies can reduce the uncertainty related to human resources and thus the risks of exceeding budgets. However, there is IT supporting positions needed instead.

Another important aspect of technology is its continuous reduction in costs. For companies, the cost benefit comparison between hiring an employee or implementing technology can easily lead the sponsors to consider technology as more cost effective than adding human resources. According the New York Times, the price advantage of machines contribute to the "reason hiring has been so sluggish".[15]

Future Outlook

Features and Functionality

Below is a list of features and functionality that we believe to see in Personal Assistant Technology within the upcoming years. Many of these features have already been reported to exist or are being released sometime in the next year in Apple's Siri, Samsung's S Voice, Google's Voice Actions, Microsoft's Bing, Iris, Nuance's Dragon products, Vlingo, and more. [16][17][18][19][20]

  • Better Language and Accent Support: The ability for VPAs to be used in multiple languages as well as better accent recognition.
  • Language Translation: The ability for VPAs to translate one language to another via speech or text input. The VPA would be able to output the translation in text and speech.
  • Connectivity to Other Apps: The ability for VPAs to control more than just certain built-in functions in mobile phones but also third-party mobile applications as well as external hardware and appliances.
  • Personal Voice Recognition: The ability for VPAs to detect and respond to only their owner's voice.
  • Faster Response Time: The ability for VPAs to be actively listening for commands rather than being activated by pushing a button. Also, this function refers to the ability for VPAs to fetch information faster than current speeds.
  • Automatically Deliver Important News/Notifications: The ability for VPAs to filter and deliver important, relevant, and timely news and notifications amid all the information overload.
  • Manage Priorities: The ability for VPAs to learn their owner's habits in order determine the order of which tasks in the day are most important and will automatically re-allocate events and tasks based on urgency.
  • Location Awareness: The ability for VPAs to be aware of their owner's location in order to make more localized and personal recommendations as well as schedule events, reminders, or tasks to occur in specific areas.
  • Determine User Preferences: The ability for VPAs to learn the preferences of their owner over time in order to make more personalized and informed decisions.
  • Personal Recommendations: The ability for VPAs to make recommendations based on aggregated preferences of similar users
  • Automatically Send Messages: The ability for VPAs to automatically send out emails, text messages or phone calls informing callers that they will be reached at a later time to answer their questions or the informing others that they are running late from another appointment.
  • Automatically Share Content: The ability for VPAs to automatically share media content it knows its user will like to various individuals or through various social networks.
  • Optimize Lifestyle Choices: The ability for VPAs to help their users meet their health, fitness, or lifestyle goals faster by optimizing certain behaviors and choices.
  • Monitor Healthcare: The ability for VPAs to carefully track their owners to ensure they are taking the most proactive and reactive steps possible to achieve optimal levels of health.
  • Education: The ability for VPAs to supplement or even replace teachers as they would be able to teach us whatever we wanted to learn all at our own pace.
  • Personal Finance: The ability for VPAs to help manage their owners manage their cash flow and investments to make sure they are maximizing their owners financial goals.
  • Navigation: The ability for VPAs to help their owners automatically know where they are going or even better, integrate with their owners vehicle to automatically take their owner where they want to go without any effort from their owners.
  • Automatically Make Purchases/Reservations: The ability for VPAs to save you the trouble of price matching, reviewing product reviews, and checking out at the counter or online and doing that automatically for you once you know what you're looking for. The other function is to have the VPAs make reservations at various places its owners have determined they want to go or based on predicting their preference.
  • Asks For Clarification: The ability for VPAs to ask clarifying "did you mean this" type questions while the technology is still learning how to interpret semantic language or when it recognizes that its owner is being too vague.
  • Authentication: The ability for VPAs to automatically authenticate their owners to access reservations, personal information, or to report to government entities.


Below are a list of technologies that we anticipate will one day take advantage of Personal Assistant Technology. Currently you'll find this technology on personal desktop and laptop computers, mobile phones, and tablets but even test platforms are in their infancy as the VPAs available today have a limited range of simple features and functionality.[21]

  • Watches: "Smart watches" are increasingly becoming mainstream (as seen with the Pebble Watch[22]) as they will pair and synchronize with other technology such as smartphones and health care devices so they can deliver notifications and monitor the status of your devices. In the future, we'll likely see these smart watches have the ability to do more and actually be a primary interface with people's VPAs.
  • Jewelry: Bracelets, necklaces, and ear rings are possibly another way for people to interact with their VPA in the future with bracelets being the closest as it resembles functionality described in the "Watches" section above.
  • Glasses: "Smart glasses" have also taken off in mainstream media (as seen with Google Glasses[23]) as they will provide visual and audio information in your line of sight, offering features such as augmented reality, image search, respond to voice commands and more.
  • Ear Pieces, Ear Buds, Head Phones: Bluetooth headsets are already a popular way for people to interact with their phones. These ear pieces can essentially pair and synchronize with other technology or stand alone, providing audio assistance and notifications that can be control with voice.
  • Headwear: Another possibility where we might see personal assistant technology are in head gear such as hats or even helmets.
  • Bags: Bags could be a tool that interacts with VPAs where they might be able to use to bag as a way to interact and receive feedback around the users surroundings to learn more about the user or remind the user of things they've forgotten.
  • Clothing: Clothing would provide a similar interface as bags for VPAs.
  • Home Appliances: Home appliances such as thermostats, televisions, ovens, refrigerators could all one day be embedded to connect with VPAs where they might be able to control all of your home appliances with voice commands and also provide notifications about your home while you are away.[24]
  • Cars and other Vehicles: A car is a great place where users can interact with their VPAs hands-free while driving. It is also a very personal space where users would be comfortable interacting with their VPA in private.[25]

Data Input

Below is a list of ways that data can be inputted into a VPA to help them interact with the world. Currently text and speech is used to control and provide initial data input but we believe this list below will eventually be added as ways for VPAs to sense their environment.[26]

  • Tone and Pitch Recognition: Currently VPAs do not have a way to detect pitch and tone to sense the feeling or sentiment of what is being inputted. One day the technology will be able to recognize the patterns in a person's pitch and tone when they speak so that VPAs can sense emotion and implied meaning.
  • Images and Video: Visual input will allow VPAs to see information on another dimension and provide information on what is being shown to them. Using this visual data, VPAs can bring up geographical information and perform visual searches without users having to type a name what they are looking at.

Convergence of VPAs and Assistive Technology

As VPAs become more and more advanced, we believe we will begin to see a convergence where VPA technology would be used heavily to develop specialized solutions to aid with the disabled, hearing impaired, visually impaired, and more. These solutions would use VPA technology to aid with things such as text-to-speech using a mobile camera, voice controls to perform simple tasks on the phone, have the phone verbally read out incoming emails and text messages, or even use the built in microphone in the mobile device to provide real time speech to text.[27] Apple has already produced a Siri commercial that features Siri reading and replying to send text messages using voice dictation for someone who is visually impaired, demonstrating the beginning of mainstream convegence.[28]

Personal Assistant Technology in Pop Culture

VPAs have made numerous appearances in popular culture, such as in TV, movies, and other mediums. People have envisioned worlds where humans and highly intelligent computers interact. Some of the most popular examples of this are shown below:

Year Title of Movie/TV Show Character Photo
1951 The Day the Earth Stood Still Gort
1956 Forbidden Planet Robbie the Robot
1968 2001: A Space Odyssey HAL9000
1977 Star Wars Trilogy C3P0
1986 Short Circuit Johnny 5


  1. It’s Technical, Dear Watson
  2. Memorial Sloan-Kettering Cancer Center (MSKCC) and Watson Partnership
  3. Watson in Health Care
  4. Memorial Sloan-Kettering Cancer Center (MSKCC) and Watson Partnership
  5. Watson in Finance
  6. Citi and IBM Enter Exploratory Agreement on Use of Watson Technologies
  7. What are the disadvantages of an automated attendance telephone system
  8. Gartner Says Organizations That Integrate Communities Into Customer Support Can Realize Cost Reductions of Up to 50 Percent
  9. Why Workers Are Losing the War Against Machines
  10. Analytics in 40 years: Machines will kick human managers to the curb
  11. The Information Explosion: A Brief History
  12. An Overview of Business Intelligence
  13. How Are Personal Digital Assistants Used in Retailing
  14. How is your company tracking the biggest expense - People
  15. Man vs. Machine
  16. TechCrunch: the Future of the Virtual Personal Assistant
  17. iDownloadBlog: Nuance Exec on Siri
  18. VentureBeat: AI Mobile Device Search
  19. Computer Weekly: Voice Controlled Mobile Banking is the Future
  20. Wikipedia: Star Trek Communicator
  21. OSX Daily: Future of Siri - Starting Cars and Adjusting Therostats
  22. Pebble Smart Watch
  23. TechCrunch: Sergey Brin Demos Google Glasses at I/O
  24. Tech Radar: Gartner - Gestures and Voice Control Won't Take Over the TV
  25. LA Times: Siri-like Service For Your Car
  26. Gotta Be Mobile: The Voice Controlled Future - Your Gadgets Are Listening
  27. Cellphones for Blind and Visually Impaired
  28. Commercial: Introducing Siri
Personal tools