Help Create the “Koko App”

The Gorilla Foundation is looking for the right tech partner to help us develop the Koko App — an educational tool designed to apply interspecies communication (via sign language) to save gorillas from extinction, improve their lives in captivity, continue our ground-breaking dialogue with great apes,  and inspire children of all ages to become scientists and voices for Nature.  Please watch the following video to learn more, read the details below the video, and contact us if you’re interested in being a partner, by emailing [email protected].

 

 

Who is Koko and What is the Koko App?

 

Koko is perhaps the world’s most famous gorilla! Known mainly for her ability to communicate in sign language (she learned over 1000 signs, and understood even more spoken words), and her gentleness with (and empathy for) other species (such as cats, birds, butterflies and humans)  she almost single-handedly changed the paradigm of gorillas from “King Kong” to “Koko’s Kitten.”

Though Koko passed away in 2018, our 5-decade interspecies communication study with her produced an immense multimedia research archive.  The Koko App is designed to apply this rich archive to benefit future generations of humans and other great apes.  In particular, it will enable humans and gorillas to learn sign language from Koko herself (an inspiring teacher) and to apply this language (which is valuable in its own right) to interpret Koko’s videos and books in ways that will expand interspecies empathy, promote conservation, and continue our dialogue with other animals.

The Koko App is designed to transmit Koko’s legacy of “conservation through communication” by teaching humans to communicate with gorillas like Koko (and each other) in American Sign Language (ASL) — with Koko as their teacher — as well as using gorilla natural gestures.  It also opens a window into the gorilla mind that, with the help of compelling video content collected over 5 decades, will increase interspecies empathy and enable us all to continue our dialogue with great apes.

We have a preliminary design for the app, as described below, but need help with the final design and implementation.  The goals and design requirements should be clear from the above video and the following feature descriptions, but let us know if you have any questions (email [email protected]).

Koko App Design Features

The following design mockup images indicate the kind of functions and features we’d like to offer in the KokoApp.   Each of these images represents a short video clip — from about 5 seconds each for individual sign demo clips, to about 30 seconds for Koko-and-Penny dialogue videos.

1) Learn Signs Directly from Koko (and Penny)

Look up hundreds of Koko’s favorite signs by name (eg, EAT, DRINK, MORE, LOVE, SORRY, THANK YOU …) and watch her demonstrate them in short, video clips. Alternatively, click on Penny’s icon and watch her and a native signer, Darlene, demonstrate how the sign is done in strict ASL (American Sign Language) compared with how Koko signs it (GSL).   

Currently, we have over 1000 of Koko’s signs demonstrated in video by Penny and Darlene, but only 100 or so demonstrated by Koko.  As the Koko App evolves, we will be adding more and more Koko demo videos.  

 

 

 

 

2) Explore the Mind of a Gorilla (via Koko and Michael)

We have hundreds of videos of gorillas Koko and Michael signing with their caregivers (especially Penny) and occasionally with celebrity visitors (like Robin Williams).   These videos demonstrate the many facets of Koko’s (and Michael’s) personality, and how they use sign language to communicate their thoughts and feelings.  Sign language thus offers us a window into the mind of a gorilla, and this treasure of research videos offers not only a way to get to know Koko (and Michael) as a “person,” but also a great tool to help you learn sign language — by watching the videos with and without subtitles.

 

Michael’s videos are more rare than Koko’s (for now, though we are in the process of digitizing many hours of early footage that should reveal much more of Michael) but when he does communicate, it’s usually about something fairly intense, and often in the form of a story.  For example, in the clip where he’s asked if what he remembers about his mother, he tells a shocking story about witnessing his mother killed by bushmeat hunters, when he was just an infant living in Africa (before he was rescued).

 

Both Koko’s story about the kitten she loved and lost, and Michael’s story about witnessing his mother’s death, show that gorillas are much more like us than we previously thought possible.  And this naturally leads to expanded empathy for gorillas as a species, which can, in turn, engender more support for both gorilla conservation and more humane captive care in zoos.

So these videos have the power to change hearts and minds about gorillas, and thus we are extremely excited about packaging them within the framework of the Koko App, as this will make it possible for citizens of gorilla habitat countries (in Africa) to see them for the first time — due to the exponential growth of cell phone use in AFrica.

In addition to videos, the Koko App will also include all of The Gorilla Foundation’s books about Koko and Michael, in the form of eBooks, with links to relevant signing videos, which which will also reinforce sign language acquisition.

 

3) Help Interpret Project Koko Videos (Citizen Science)


The Gorilla Foundation began working with Koko in 1972 and Michael in 1974, and has accumulated nearly 5 decades of interspecies communication research videos that amount to over a thousand hours of footage — most of it un-reviewed.

This is an excellent opportunity for crowd-sourced translation of Koko’s sign language videos — which could take us years to complete otherwise.   

Sign language translation of Koko’s (and Michael’s) videos does not require any prior knowledge of sign language, and in fact, it’s a good way to learn sign language.

The Koko App features 3 ways to look up signs:  by a) Sign Name, b) Category, and c) Description. 

When you don’t know the name (or category) of a sign that you’ve just watched, you can look it up by Description.  When you search for a sign by Description, you can describe the sign via up to 3 standard parameters:  Hand Shape, Movement and Location.   So, for example, suppose you saw Koko signing something like what’s in the image on the left (within a video).  You could then simply search for all Koko signs “Located” at her Mouth, as on the right.  This would bring up a small subset of signs (which would be even smaller if you added a description of the Hand Shape and/or Movement) in your search results. 

You then could quickly identify which of the found signs looks most like what Koko is signing on the left.  And in the process, you would have “learned” the sign for (say) “Frown” and wouldn’t have to look it up by Description the next time, you could just search for “Frown” by “Name” to confirm you remembered it correctly (and revert to Description if you’ve forgotten the sign).

Note that while this process can be very accurate and fast in principle,  it may require some refereeing by Gorilla Foundation staff (and/or AI) because we currently have an incomplete set of individual-sign clips of Koko (though we have all 1000+ video clips of Penny demonstrating the signs).

By opening up the entire Koko Video Archive to crowd-sourcing through the Koko App, we can continually update the app with both new sign demos, and new gorilla videos (with and without sign subtitles) to enrich all of the other features of the app (learning to sign, exploring the mind of a gorilla, translating signs, and conservation)!

The KokoArc Video Archive Spans5 Decades of Interspecies Communication

4) Translate Zoo Gorilla Natural Gestures (improve their lives)

Gorillas have a natural capacity for gestural communication.  Zoo gorillas have been observed using over 100 natural (untaught) gestures consistently in both the US and the UK.  However, we haven’t yet been able to translate many of these gestures, and learning them could create a huge advantage for gorilla care — as their caregiver would be better able to understand their needs and wants without having to teach them a “new ” language  such as ASL.

For example, the gorilla at left (JJ at the Miami Zoo) was having a rough day.  It looks like he was trying to say:  “Help, no more help” in response to a bunch of visitors attempting to throw food at him.   By adding more context, experience and data, it may be possible to translate a large number of such gestures.

Here’s how the Koko App can help.  Allow users to upload videos that they either take themselves at zoos, or capture from the internet, and ask others to submit their “guesses” (from context) as to what the gorillas are saying.  Then combine expert (relevant zookeeper) inputs with a deep learning algorithm to find the best match for each word or phrase.  As we develop a large enough database of natural gesturals, it will become easier to translate new footage with existing gestures, or to identify new gestures to translate.

We can implement this process in 2 different ways:  automatic uploads of videos and translation guesses, and/or refereed submissions, selectively uploaded for the public to see.  In either case, we could add regularly scheduled Zoom video panel discussions to discuss status and give tutorials on how to participate in the process of learning gorilla natural gestures.  We can also provide updates from specific Zoos on how expanding our shared language with gorillas is improving their care and enrichment in captivity.

Finally, note that the Koko App can also be used to teach basic sign language (ASL) to gorillas (by zookeepers who wish to participate).  Establishing just a small shared ASL vocabularly could create a “bridge” to learning more of gorillas’ natural gestures, as they would be able to define them for us in a language that we already know (ASL).  Koko used to create compound signs from the ASL signs she knew to stand for things for which she did not yet know the sign.  For example she compounded the signs for “finger” and “bracelet” into “finger-bracelet” to represent the sign for “ring” (which was never taught her).  This shows that gorillas have the logical capacity for creating “synonyms” and could theoretically create ASL synonyms for some of their natural gestures,  in order for humans to be able to understand.  When we told Koko we didn’t understand something she was signing, she would often come up with alternate ways to say it until we finally did get it.

 

5) Help Save Gorillas from Extinction (Conservation via Communication)

 

 

All of The Gorilla Foundation’s books — for both children and adults — will be available as “eBooks” within the Koko App.  Not only will you be able to read these books for free in a convenient navigational environment, you’ll also be able to click on sign words (within interspecies dialogues) that appear in these books,  and bring up a popup video teaching you (or reminding you of) the sign.

The initial release of the Koko App can include all of the following eBooks, with selected links to individual signs and related videos:

  • Koko’s Kitten (ages 2-12, and 21+)

  • Koko’s Story (ages 2+)

  • A Wish for Koko (ages 7+)

  • Michael’s Dream (ages 12+)

  • The Education of Koko (ages 15+)

 

6) Learn About Conservation from Gorilla Michael (a bushmeat orphan)

 

Because of the skepticism people often have around interspecies communication, videos are often more powerful than books in overcoming the credibility gap.  This is especially important in Africa, where we have distributed Koko’s books to students and poachers in gorilla habitat countries, but have not (until now) been able to show them what we mean.  Due to the recent mass adoption of cell phones in Africa, we can now overcome this hurdle where needed most.

The Gorilla Foundation

The Gorilla Foundation (aka Koko.org) was established as a nonprofit foundation (501c3) in 1976 to explore interspecies communication with gorillas for the benefit of all great apes, through science, education and direct conservation efforts.    We have tens of thousands of supporters (donors) who resonate with our mission of Conservation through Communication.

It is not commonly known that “all gorillas are Kokos”!  They all have natural gestural abilities (their own intrinsic sign language), which makes it easy for them to learn American Sign Language (ASL), and through interspecies communication with gorillas Koko and Michael we have learned how incredibly intelligent, creative and emotional gorillas are.  The longer we communicate with them, the more we will learn,  about them and ourselves. 

Our supporters want us to bring Kokos Legacy to fruition; but we can’t do it alone; we need partners whose expertise is complementary to ours.

 

 

 

Our Ideal Tech Partner

The ideal partner, can be either a company, a small group of volunteers, or an exceptional individual, with one or more of the following skill sets:

  • Successful large-scale App Development experience
  • User-Interface Design expertise for a wide-spectrum user audience
  • Data mining technology, including AI (deep learning) to recognize meaningful hand gestures and facial expressions in videos, and to translate handwritten research notes.

Ideally, this would be a pro bono (philanthropic) effort initially, with the possibility of sharing revenue if we decide to add subscription or upgrade options to the Koko App.

 

 

Contact Us:

Call or email Dr. Gary Stanley, COO of The Gorilla Foundation at [email protected] or 650.722.6810.  We are located in Northern California.

Let us know some convenient times to meet via Zoom and explore the opportunities.

7) Upload and Share Videos of Zoo Gorillas to Learn their Natural Gestures