The Google Assistant SDK adds support for additional languages & more

Google announced it has expanded the Google Assistant software development kit to support additional languages. That means developers can now bring Google Assistant applications to more people. Google Assistant now supports these additional languages and regions English Australia, English Canada, English UK, English US, French Canadian, French France, German and Japanese.

Lack of support for languages can impede development on the platform. For example, my company has been trying to find ways around language barriers to build Jewish apps, but Hebrew is not yet supported. The difficulty is having the Google Assistant APIs understand the language or regional language dialects and respond with a proper answer. So in this example, if someone asks what time is mincha, which is afternoon services in the Jewish world, Google cannot understand the word “mincha” because it is a Hebrew word. Bringing more support for additional languages and regions helps Google expand the ecosystem of the Google Assistant platform.

Other improvements to the Google Assistant SDK include more customized settings, including changing the device’s language, location and nickname and enabling personalized results. The API now also supports text-based queries and responses. Developers can also utilize the new Device Action functionality to build Actions directly into your Assistant-enabled SDK devices. Also, new APIs allow developers to register, unregister and see all devices that you have registered for better device management support.

Google bringing the Assistant to tablets and Lollipop Android phones

Google is rolling out the Assistant to more devices. It will soon be available on Android tablets running Nougat and Marshmallow, and smartphones running Lollipop.

Tablets in the US running English will be the first to get access. However, a wide array of Android 5.0 smartphones (Lollipop) will get the Assistant: Those operating in English in major markets and in Spanish in the US, Mexico and Spain; and Lollipop smartphones in Italy, Japan, Germany, Brazil and Korea.

Google is pushing the Assistant out to more devices as the market becomes more competitive and AI development accelerates.

A July 2017 report from Verto Analytics found that 42 percent of US smartphone owners used virtual assistants, in the aggregate, on average 10 times per month. That translated into more than 70 million smartphone owners and almost 1 billion hours per month in the US. The numbers are likely somewhat higher now.

Personal Assistant Usage Numbers & Demographics

Source: Verto Analytics (5/17)

Siri was the most used (largest audience), but Cortana and Alexa were the fastest-growing assistants, according to Verto.

Separate research has found that virtual assistants are used much more frequently on smart speakers, which makes sense because of the general absence of screens: almost three uses per day vs. less than one for smartphones.

Google Lens an impressive start for 'visual search'

Google Lens has gone live or is about to on Pixel phones in the US, the UK, Australia, Canada, India and Singapore (in English). Over the past couple of weeks, I’ve been using it extensively and have had mostly positive results — though not always.

Currently, Lens can read text (e.g., business cards), identify buildings and landmarks (sometimes), provide information on artwork, books and movies (from a poster) and scan barcodes. It can also identify products (much of the time) and capture and keep (in Google Keep) handwritten notes, though it doesn’t turn them into text.

To use Lens, you tap the icon in the lower right of the screen when Google Assistant is invoked. Then you tap the image or object or part of an object you want to scan.

As a barcode scanner, it works nearly every time. In that regard, it’s worthy and a more versatile substitute for Amazon’s app and just as fast or faster in many cases. If there’s no available barcode, it can often correctly identify products from their packaging or labels. It also does very well identifying famous works of art and books.

Google Lens struggled most with buildings and with products that didn’t have any labeling on them. For example (below), it was rather embarrassingly unable to identify an Apple laptop as a computer, and it misidentified Google Home as “aluminum foil.”

When Lens gets it wrong it asks you to let it know. And when it’s uncertain but you affirm its guess, you can get good information.

I tried Lens on numerous well-known buildings in New York, and it was rarely able to identify them. For example, the three buildings below (left to right) are New York City Hall, the World Trade Center and the Oculus transportation hub. (In the first case, if you’re thinking, he tapped the tree and not the building, I took multiple pictures from different angles, and it didn’t get one right.)

I also took lots of pictures of random objects (articles of clothing, shoes, money) and those searches were a bit hit-and-miss, though often, when it missed it was a near-miss.

As these results indicate, Google Lens is far from perfect. But it’s much much better than Google Goggles ever was, and it will improve over time. Google will also add capabilities that expand use cases.

It’s best right now for very specific uses, which Google tries to point out in its blog post. One of the absolute best uses is capturing business cards and turning them into contacts on your phone.

Assuming that Google is committed to Lens and continues investing in it, over time it could become a widely adopted alternative to traditional mobile and voice search. It might eventually also drive considerable mobile commerce.

Google brings local lead generation to Google Assistant and Google Home

Google is bringing new forms of local search to the Google Assistant and Google Home. The company announced it’s working with local home service providers “like HomeAdvisor and Porch.”

On any platform where Google Assistant is available, users will be able to ask for contractors (e.g., “Ok Google find me a plumber”). That initiates a structured interaction which generates a lead or contact with a local service provider.

In the case of IAC-owned HomeAdvisor, which now also owns Angie’s List, users can ask to be connected by phone at the end of the process to a contractor or receive a list of relevant, pre-screened contractors. The following graphic depicts part of the user experience and the structured Q&A that’s used to refine the lead.

This is a highly structured local search and lead-generation experience that will bypass conventional search results (i.e., business listings). Google said the new functionality would be rolling out in the next week or so.

Google itself offers local lead generation for contractors and service providers with Local Services ads that appear in search results. What’s unclear is how providers from HomeAdvisor, Porch (and perhaps Google) will be prioritized or presented for a given query.

In its blog post, Google didn’t say anything about its own advertisers or how many third-party directories might eventually be involved. There’s also no word on whether Google will collect a share of the lead price or any sort of “toll” otherwise.

Currently, if you ask Google Home for a local service provider (e.g., “I need a house painter”) you’ll get three “thin” listings with address information but little else. The coming experience will supplant that, offering a more personalized result based the specific request and subsequent information provided.

Because it’s not yet live, we don’t know how well it will work. It has the potential to be effective both for the consumer and the contractor. Generally speaking, this is going to be bottom-of-the-funnel activity.

.mktoButton{background:#000!important;}

Stay up to date on local search, digital assistant-related and other search marketing news. Sign up for our newsletter below.

Report: Smart speaker owners increasingly using them instead of typing or swiping

Unlike VR headsets or wearables, smart speakers are rapidly emerging as a mass market technology platform. The latest to document relatively high satisfaction and usage of these devices is call-tracking and analytics company Invoca.

Earlier this year, Invoca surveyed 1,000 people in the US who own an Amazon Echo or Google Home device. The survey asked questions about current behavior and a range of hypothetical scenarios.

The company found that people use smart speakers more frequently over time, with 89 percent using them daily. Here’s a more detailed usage breakdown:

33 percent of owners said they used the devices more than five times daily.28 percent used them four to five times a day.24 percent used them two to three times a day.

In addition, 58 percent of respondents said that they used assistants to “accomplish tasks they used to do through typing or swiping.” So there’s some substitution going on, and there’s apparently an appetite for more.

By some estimates, Amazon controls 70 to 75 percent of the smart speaker market in the US. But the Invoca survey found that people were open to devices from other companies not yet in the market, especially Facebook. Nearly 60 percent (58 percent) of survey respondents said they would potentially buy a voice assistant from the social network if it sold one. (Facebook is rumored to be working on such a device.)

Invoca also reports that 73 percent of respondents said they’d made a purchase through a voice assistant, and 39 percent said interaction with these devices had influenced purchases made elsewhere. (I’m skeptical of the accuracy of the 73 percent figure.)

One of the more interesting findings: Invoca also discovered some receptiveness to ads on these devices, provided users had a degree of control or they were personalized in some fashion. Currently, there are no ads on Google Home or Alexa devices, though promotional audio messages are likely to arrive at some point. Earlier this year, Amazon shut down such an attempt by third-party analytics company VoiceLabs.

The report also explored tasks or activities that were being pursued by device owners in several vertical contexts: banking, travel and healthcare. Here were the top three smart speaker objectives or activities reported in each category:

Travel

    Inquire about hotels.Check flight status.Purchase or book something.

Banking

    Check a balance.Pay a bill.Track spending.

Healthcare

    Ask about symptoms.Ask about health/diet tips.Connect with hospital or doctor.

Survey respondents were not without some criticisms and complaints, mostly having to do with being comprehended, the length of conversations they could have and the complexity of tasks they could execute.

About 90 percent of owners said that if they’re unable to get what they’re seeking from the smart speaker they turn to search. And 76 percent said that they would like to be able to connect with a human through the device, as a backup, if it was unable to answer their question.

After the smoke from holiday 2017 clears, we could discover as many as 50 or 60 million smart speakers installed in US households.

Google aims to make apps for Google Assistant more functional and discoverable

Today Google announced a number of updates and improvements to help make third party apps for Google Assistant more functional and easier for users to discover. It’s also adding new capabilities for developers.

Developers can now build Assistant apps in Spanish, Portuguese and Indian English. For the UK, Google announced the availability of transactional capabilities (purchases, reservations or appointments). For the app directory, Google is adding new sections (what’s new and what’s trending). And there will now be an autocomplete feature to help suggest apps to users.

Right now the app directory is relatively hidden and I suspect only a tiny percentage of users know it exists. However, within the directory Google is also creating new subcategories for apps that are more task-specific. The company uses the example of “Order Food” or “View a Menu” in the category “Food & Drink.” There will also be new badges for family friendly apps.

Google is also trying to facilitate more natural app discovery — and this is probably going to be more common than other methods — through “implicit invocation” or “implicit discovery.” This is a form of app recommendations, when Google believes that a specific app can answer a user question or fulfill a need, or when the user’s verbal command is the “action phrase” for the Assistant app itself:

Implicit invocation occurs when users invoke your app without using its name. This type of invocation occurs when users tell the Google Assistant to do something that’s similar to the action invocation phrase for one of your configured intents, or when the user is in a context where your app would be appropriate.

There’s a very SEO-like (“voice optimization”) approach to getting your app recognized and invoked by Google in this “implicit” scenario.

Most compelling of these new features is the introduction of what Google calls “multi-surface conversations.” This is essentially Google Home send to phone.

An interaction begins on a Google Home device and is then sent to a smartphone for completion. The example Google provides is food ordering that begins on Google Home, with the payment transaction concluding on a smartphone. For relatively obvious reasons, this could be a boon to travel and e-commerce transactions on Google Home.

Google is also enabling personalization (preferences), daily updates, notifications and directory analytics.

Integrating third party capabilities into the Google Assistant has the potential to be extremely useful. Apple’s Siri is doing this and, obviously, so is Amazon Alexa with “skills” (most of which are currently worthless). But the problem of “app discovery” is even more acute in the voice context than it is in smartphone app stores.

This is why “implicit discovery” holds so much promise. But Google can also do a great deal more to surface developer apps and content within the Google Assistant and online.

Voice search: Content may be king, but context is queen in the new voice-first world

In 2016, Google said that 20 percent of all mobile queries were voice searches. Since that time, the number of virtual assistants in US households has continued to swell, with tens of millions of voice-enabled home devices projected to be in use.

Voice as a primary search interface — beyond mobile phones — is a reality. Marketers need to rapidly iterate on their mobile-first strategies, to adapt to the voice-first marketplace. And as the aptly titled e-book released today [registration required] suggests, voice search changes everything.

I sat down with the book’s author, Yext VP of Industry Insights Duane Forrester, to discuss the landscape of voice search, how it will impact the business of search marketing, and what marketers can do to prepare for this evolution in search user interfaces.

“Voice engagement is the most likely scenario that will challenge the biggest players in search for supremacy.”

Michelle Robbins: What inspired you to put this e-book together?

Duane Forrester: The work we do at Yext is focused on helping businesses understand what data they can control, and empowering them with ways to manage that data. So from that point of view, there was a lot of support for exploring this developing space. Personally, I’ve always been an early adopter. The last decade of my life I’ve been fortunate enough to see the leading edge of technology up close and interact with it personally, so as “voice” developed to what we have today, I’ve been engaged and watching its progress.

MR: The major players in the space have been established. Do you see room for any other competitors to enter the voice arena?

DF: Absolutely. There is a boom happening in China right now with dozens of new companies entering the smart speaker space. While most won’t survive, it’s inevitable we’ll see new devices reach our shores next year, driving prices down and adoption up.

Most of that expansion will be white-labeled products (Google Assistant built into a Samsung TV, for example), but from the consumer’s point of view, it’ll be less about buying because of the embedded assistant and more about brand awareness around specific products. People don’t buy the Samsung TV because of Google Assistant (or Siri, or Cortana, or IBM, etc.), they buy it because Samsung makes excellent televisions. The voice assistant is a nice addition. That’s our immediate future. Over time, however, this could change if one or more of the leaders make significant technology breakthroughs that bring obvious differentiation and improvements.

MR: Is there anything holding back even greater adoption of voice-enabled devices?

DF: We’re starting to see the end of people’s reluctance to speak to their devices. This was a major factor in adoption over the last five years. Couple that with less than stellar services and results, and adoption was predictably sluggish — right up until Amazon landed in millions of living rooms around the world.

The biggest factor in voice adoption remains time. As services surpass an accuracy rate of 98%+ and consumer upgrade devices, or have their first contact with new devices that are voice enabled, the growth will continue. Voice will conquer all.

MR: How can marketers, and search marketers in particular, shift from a ‘content is king’ focus to competitively prepare for the ‘context is queen’ world and surface as the one primary voice result?

DF: The beauty of this is clear. All the investment that’s been poured into content continues to pay dividends in a voice-first world. If anything, in order to truly get to the context-first scenarios we have today, you need deep, detailed, rich content. But even here, context plays a role. If the request is for the temperature, the platform being engaged will determine location as part of the relevancy factoring. The answer (let’s say “72 degrees”) in any other context might seem “thin” by nature. But as an answer to “What’s the temperature outside?,” it’s a perfect fit.

A more complex scenario might look like “Who is Harry Potter?” and “What is Harry Potter?” The former should bring back an answer about a fictional person, while the later should elicit a response about a fictional series about a boy wizard, etc. The answers for the latter would be deeper, and pull from richer “answers” provided by websites.

To be included in the “spoken answer” column, we have no set best practices from the engines to follow, but we do have some common best practices we know they respond to for things like the Answer Boxes. And increasingly, it’s those answer box contents that are being spoken aloud to inform consumer queries.

As for specific tactics people can employ, here’s a short list. This is in addition to the usual quality content production and SEO best practices.

    Adopt a long tail/conversational phrase approach to targeting what to produce content around.Build out detailed answers to common (and even uncommon) questions related to your products and services.Use Schema to mark up your content (where appropriate).Clean up your own house — be sure crawlers can find your content.Make sure your site is mobile-friendly — not really an option these days.Make security a priority — becoming more of a trust signal.

“A picture is worth a thousand words”

MR: What additional innovations in voice are coming into play?

DF: If you’ve shopped via a voice device, you’ve encountered an area that will improve significantly when visuals are added. Ask the system to buy a blue sweater, and you immediately realize without being able to see the sweater, you’re missing a lot of information needed to make an informed purchase.

This is where visual search comes into play, and it’s here now as the logical next step from voice search. We see initial products from Amazon in the market now (Show and Spot), and I expect to see more companies fielding visually-enabled voice devices soon. In terms of e-commerce, this expands the usefulness of current content investments like product videos.

MR: What kind of technology investments should marketers be making to address this new playing field?

DF: Things that were optional even just a couple years ago, are no longer optional. Being mobile-friendly is a requirement. Being secure is rapidly becoming a differentiator. Marking up your content is no longer a nice-to-have. Every day adoption of those technical items grows, which means the playing field is changing. If a search engine suggests a protocol is worth using, it’s worth paying attention.

Things like Schema markup help an engine grow trust in your website and content, so take advantage of that. Being secure shows an investment in protecting consumers, again an area the engines favor and actively support. And if you really want to walk a mile in your customer’s shoes, to really learn what their journey is like, you’ll buy the main voice-enabled devices on the market today. Set them up and use them all day, every day. This practice will uncover new features and highlight new opportunities for you to align with the customer’s journey.

MR: What kind of personnel investments should organizations be making to effectively compete in a voice-first world?

DF: It’s highly likely that a business already has the skill sets they need on hand. If they have an SEO person or team, they’re off to an excellent start. To truly take advantage of new environments like voice and visual search, though, you need to have someone who has a broad understanding of emerging opportunities, has the reach to influence across and within your company, and can offer guidance based on experiences in discrete areas. That’s the role of a Digital Knowledge Manager (DKM).

The DKM can help ensure all assets in a company are aligned to best effect, while also keeping the company up to speed on emerging technologies. From the top, it’s the DKM that guides. From a more tactical level, it’s likely a technically proficient SEO aligning efforts across research, content development and deployment. That combined effort can help a company get started and take a leadership position in their verticals.

Join us at SMX West this March in San Jose, where we’ll feature industry leaders sharing tips and tactics for search marketing success in voice search, local and mobile SEO and much more!

.mktoButton{background:#000!important;}

Stay up to date on voice search and other industry news and trends.

Google Assistant now offering a wide range of games for kids and families

The battle of the smart speakers and home assistants is in full swing. And both Amazon and Google think that gaming and fun will help provide a competitive edge.

Amazon introduced Echo Buttons, which enable families to play Alexa-based games together, in September. Today Google announced a trove of games for families and kids: “[T]he Google Assistant now has more than 50 new games, activities and stories designed for families with kids.” They include trivia, musical chairs, storytelling and more.

Games for Google Assistant are available on Home devices, smartphones and other devices where the Assistant is available. This is also where Google seeks to compete, as a platform across more devices (“ambient computing”) than Amazon can offer.

Google has also made it possible to personalize the Assistant for kids under 13. Home devices can recognize up to six different voices. Accordingly, kids can use the same devices as their parents, but the Assistant will recognize the child’s voice and offer different options and experiences.

Parental controls are powered by “Family Link.” It’s an app that gives parents the ability to manage their kids’ Android device experiences.

Target expands voice-commerce relationship with Google to battle Amazon

Google announced nationwide expansion of its Google Express relationship with Target. Users in the Continental US will now be able to buy from Target through the Google Assistant and receive Google Express delivery. The voice-commerce relationship extends to Google Home devices.

Features and capabilities are nearly identical to those announced in August with Walmart. And while the deal isn’t yet fully operational, ultimately you’ll be able to:

order and reorder from Target with free shipping on orders exceeding $35.opt in for personalized recommendations (as an existing Target customer) and a “quick re-order experience based on past Target purchases.”shop on any device (including Android TV) where the Google Assistant is available.

Google is doing battle with Amazon on multiple fronts, and so are its Google Express retail partners. Google sees voice-based shopping from an array of branded retailers as a point of differentiation vs. Amazon. By the same token, retailers need to make themselves accessible through virtual assistants and smart speakers. There’s an alignment of interests, creating an anti-Amazon alliance of sorts.

A recent ad campaign for Google Express promotes the notion of “all your favorite stores in one place.” In addition to Target and Walmart, the service offers access to Kohl’s, Joann, Sur la Table, Walgreens, Staples, Toys R Us and numerous others.

Target is a top 10 e-commerce site, but it badly lags Amazon, and it has no voice-commerce capability today. It’s wise for Target to work with Google to leverage the latter’s distribution. However, the question in my mind is: Will Google retail partners like Target benefit in a meaningful way from these deals, or will they eventually turn into little more than suppliers for Google Express?

Echo and Home will probably have to tell you they're always listening — in Europe

A number of Google Home Mini devices that were distributed to members of the press had a defect that caused them to record everything being said around them. This discovery renewed privacy concerns surrounding smart speakers as surreptitious listening devices in our homes.

The problem was first discovered by Android Police. Once being notified, Google investigated and fixed the issue:

The Google Home team is aware of an issue impacting a small number of Google Home Mini devices that could cause the touch control mechanism to behave incorrectly. We immediately rolled out a software update on October 7 to mitigate the issue.

Who is affected: People who received an early release Google Home Mini device at recent Made by Google events. Pre-ordered Google Home Mini purchases aren’t affected.

As a general matter, Google Home and Amazon Alexa devices must “listen” to surrounding conversations to capture “wake words” (e.g.,”Alexa,” “OK Google”) that activate them. Some privacy advocates have sounded alarms about this and expressed concern that these devices could be abused by unscrupulous law enforcement or other malevolent state actors (see Orwell’s Telescreen).

In a well-publicized criminal case in Arkansas, local prosecutors sought recordings on an Amazon Echo in a murder investigation. Amazon fought to prevent authorities from getting access to these recordings without a warrant. The defendant in the case ultimately consented to the release of any stored data, so the warrant issue was never formally ruled on by a court.

As Internet of Things devices proliferate, privacy warnings about personal data collection will intensify. It’s very likely that there will be more than 30 million smart speakers in US homes by year-end. Google and Amazon are competitively discounting and aggressively marketing them. Google’s $49 Home Mini was introduced as a low-cost answer to the Amazon Echo Dot, which Amazon just discounted to be $5 cheaper than the Mini.

These devices are also widely available in Europe, which raises the question of how they will be addressed under the forthcoming General Data Protection Regulation (GDPR) taking effect in May 2018. Millions of smart speakers will be installed in European homes by then.

In order to process “personal data,” companies must obtain opt-in consent from users:

Consent must be clear and distinguishable from other matters and provided in an intelligible and easily accessible form, using clear and plain language. It must be as easy to withdraw consent as it is to give it.​ Explicit consent is required only for processing sensitive personal data — in this context, nothing short of “opt in” will suffice. However, for non-sensitive data, “unambiguous” consent will suffice.

It’s safe to say that these devices will be “processing sensitive personal data” and that explicit consent will be required in every case.

There’s no explicit mention of smart speakers in the GDPR documentation. However, artificial intelligence is addressed to some degree in Article 22 (“Automated individual decision-making, including profiling”), which says:

The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her [unless explicit consent is provided].

Most consumer-facing AI technologies, including smart speakers and self-driving cars, will require explicit opt-in consent in Europe. For Echo or Home, it might be as simple as a verbal statement played upon setup, which asks for the owner to OK use of his or her personal data. Alternatively, there might need to be ongoing or periodic disclosures and consent.

There’s currently a lack of clarity about what will be specifically required from smart speaker makers. We’ll likely see some guidance, however, from the EU or NGOs in the next several months. The consumer question will be: how do I feel about a third-party recording device in my home?