On Distributed Cognition

Nervous systems do not form representations of the world, they can only form representations of interactions with the world.[9]

The emphasis on finding and describing “knowledge structures” that are somewhere “inside” the individual encourages us to overlook the fact that human cognition is always situated in a complex sociocultural world and cannot be unaffected by it.

— Hutchins, 1995 p. xiii


From https://en.wikipedia.org/wiki/Distributed_cognition

Notes: Ontology creation for cognitive computing

The creation of ontologies continues to slow down many cognitive computing projects. Here are some notes from a quick exploration.

From Wikipedia:

“Ontology is the philosophical study of the nature of being, becoming, existence, or reality, as well as the basic categories of being and their relations. Traditionally listed as a part of the major branch of philosophy known as metaphysics, ontology often deals with questions concerning what entitiesexist or may be said to exist, and how such entities may be grouped, related within a hierarchy, and subdivided according to similarities and differences. Although ontology as a philosophical enterprise is highly theoretical, it also has practical application in information science andtechnology, such as ontology engineering.”


More interesting for our purposes from the arena of cognitive computing is the area of “ontology engineering”. Here’s the definition from Wikipedia:

Ontology engineering in computer science and information science is a field which studies the methods and methodologies for building ontologies: formal representations of a set of concepts within a domain and the relationships between those concepts. A large-scale representation of abstract concepts such as actions, time, physical objects and beliefs would be an example of ontological engineering.[2]Ontology engineering is one of the areas of applied ontology, and can be seen as an application of philosophical ontology. Core ideas and objectives of ontology engineering are also central in conceptual modeling.


This looks interesting for getting to the next level of detail (if somewhat dated (2004)):

Ontological Engineering: With examples from the areas of Knowledge Management, e-Commerce and the Semantic Web

Fragments of reviews for the book:

  • “Also discussed in the book, and of enormous practical interest, is the automation of the ontology building process. Called `ontology learning’ by the authors, they discuss a few of the ways in which this could take place. One of these methods concerns ontology learning using a `corpus of texts’, and involves being able to distinguish between the `linguistic’ and `conceptual’ levels. Knowledge at the linguistic level is described in linguistic terms, while at the conceptual level in terms of concepts and the relations between them. Ontology learning is thus dependent on how the linguistic structures are exemplified in the conceptual level. Relations at the conceptual level for example could be extracted from sequences of words in the text that conform to a certain pattern. Another method comes from data mining and involves the use of association rules to find relations between concepts. The authors discuss two well-known methods for ontology learning from texts. Both of these methods are interesting in that they can apparently learn in contexts or environments that are not domain-specific. Being able to learn over different domains is very important from the standpoint of the artificial intelligence community and these methods are a step in that direction. The processes of `alignment’, `merging’, and `cooperative construction’ of ontologies that are discussed in the book are also of great interest in artificial intelligence, since they too will be of assistance in the attempt to design a machine that can reason over multiple domains.”
  • “The automation of ontology building would of course be a major advance. To accomplish this however would require that the machine be able to simultaneously and recursively construct the knowledge base and reason over it effectively. This is a formidable challenge indeed.”
  • “A large portion of the book describes the acute problem of somehow extracting meaning in a programmatic manner from data. Because the manual making of an ontology simply does not seem to scale, given the realities of gigabyte databases. We see that there is a natural decomposition of the problem into a linguistic step and a conceptual step. The former is tied to a particular human language. The latter is the nut of the problem. Current methods look promising, but are certainly not the last word.”


Wikipedia talks about Ontology Learning:

Ontology learning (ontology extraction, ontology generation, or ontology acquisition) is the automatic or semi-automatic creation of ontologies, including extracting the corresponding domain’s terms and the relationships between those concepts from a corpus of natural language text, and encoding them with an ontology language for easy retrieval. As building ontologies manually is extremely labor-intensive and time consuming, there is great motivation to automate the process.

Typically, the process starts by extracting terms and concepts or noun phrases from plain text using linguistic processors such as part-of-speech tagging and phrase chunking. Then statistical[1] or symbolic [2][3] techniques are used to extract relation signatures, often based on pattern-based[4] or definition-based[5] hypernym extraction techniques.”


A few extracts from:

Wong, W., Liu, W. & Bennamoun, M. (2012), “Ontology Learning from Text: A Look back and into the Future”. ACM Computing Surveys, Volume 44, Issue 4, Pages 20:1-20:36.

This article is somewhat focused on building ontologies for the semantic web, but has interesting observations on the state of the art for automated ontology creation.

  • “Ontologies can be thought of as directed graphs consisting of concepts as nodes and relations as the edges between the nodes. A concept is essentially a mental symbol often realized by a corresponding lexical representation (i.e., natural language name). For instance, the concept “food” denotes the set of all substances that can be consumed for nutrition or pleasure. In Information Science, an ontology is a “formal, explicit specification of a shared conceptualisation” [Gruber 1993].”
  • Screen Shot 2016-02-26 at 12.37.44 PM
  • “There are five types of output in ontology learning, namely, terms, concepts, taxo- nomic relations, non-taxonomic relations, and axioms. Some researchers [Buitelaar et al. 2005] refer to this as the ontology learning layer cake.”
  • Screen Shot 2016-02-26 at 12.40.07 PM
  • “In document retrieval, the object of evaluation is documents and how well systems provide documents that satisfy user queries, either qualitatively or quantitatively. However, in ontology learning, we cannot simply measure how well a system constructs an ontology without raising more questions. For instance, is the ontology good enough? If so, with respect to what application?”
  • “Since the publication of the five survey papers [Ding and Foo 2002; Gomez-Perez and Manzano-Macho 2003; Shamsfard and Barforoush 2003; Buitelaar et al. 2005; Zhou 2007], research activities within the ontology learning community have largely been focused on improving (1) term extraction and concept formation and (2) relation discovery techniques. The learning of ontologies (3) from social data and (4) across different languages has also been a topic of great research interest in the later part of the past decade.”
  • “Besides the social dimension of ontology creation, ontology learning from multilin- gual text is also gaining popularity. Hjelm and Volk [Hjelm and Volk 2011; Hjelm 2009] discussed ways to automatically construct ontologies by exploiting cross-language in- formation from parallel corpora.”
  • On Scoring and Extracting Terms: “The current state of the art is based mainly on statistical semantics and paradigmatic and syntagmatic relations, that is to say, we determine the relevance of terms through observations in very large samples and through the way the constituents of a term are put together.”
  • “… for taxonomic and non-taxonomic relation discovery, we are witnessing the increasing application of lexico-syntactic patterns, association rule mining, and rules based on syntactic dependencies on very large datasets from the Web.”

Fragments from: http://www.aclweb.org/anthology/J06-4009.pdf (book introduction, 2005)

  • Ontology learning has become a major area of research within the wider area of artificial intelligence and natural language processing. This is largely due to the adoption of ontologies (especially formal ontology expressed in OWL) as the standard form of knowledge representation in the Semantic Web.
  • By a judicious selection of techniques ranging from part-of-speech tagging, chunking, and parsing to clustering and IR methodologies, they attempt to deal with the three fundamental issues involved in constructing ontologies: associating terms, building hierarchies of terms and concepts, and identifying and labeling ontological relations.
  • “ontology-learning layer cake,”

Fragments from chapter one of “An introduction to ontology learning”, Lehmann and Volker, 2014

  • Ontology learning approaches are as heterogeneous as the sources of data on the web, and as different from one another as the types of knowledge representations called “ontologies”
  • no general agreement on which requirements the formal representation needs to satisfy in order to be appropriately be called an ontology. Depending on the particular point of view, ontologies can be simple dictionaries, taxonomies, thesauri, or richly axiomatized top-level formalisations
  • Ontologies play a central role in data and knowledge integration. By providing a shared schema, they facilitate query answering and reasoning over disparate data sources
  • However, the construction of ontologies is a highly expensive task which crucially hinges on the availability of scarce expert resources [39]. In order to build a formal ontology for a particular domain of interest, for instance, specialized domain knowledge needs to be acquired and formalized in a way that automated inference will yield the expected results. This goal can only be achieved if domain experts collaborate with skilled ontology engineers familiar with the theory and practice of knowledge representation – and once the ontology has been constructed, evolving knowledge and application requirements will demand for continuous maintenance efforts [[reference 39 is: Elena Simperl, Tobias Buerger, Simon Hangl, Stephan Woelger, and Igor Popov. Ontocom: A reliable cost estimation method for ontology development projects. Web Semantics: Science, Services and Agents on the World Wide Web, 16(0):1 – 16, 2012]]
  • One grouping: Ontology Learning from Text mostly focuses on the automatic or semi-automatic generation of lightweight taxonomies by means of text mining and information extraction. Many of the methods used in ontology learning from text (e.g. lexicosyntactic patterns for hyponymy detection or named-entity classification) are inspired by previous work in the field of computational linguistics, essentially designed in order to facilitate the acquisition of lexical information from corpora. Some ontology learning approaches do not derive schematic structures, but focus on the data level. Such ontology population methods derive facts from text. A popular example is the Never-Ending Language Learning (NELL) project [10], which reads the web to add statements to its knowledge base and improves its performance over time, e.g. via user feedback.

My reading, overall, is that the creation of ontologies remains a time consuming exercise for experts and a relatively unsolved problem for automated systems.


Other references:

2008, Ontology Engineering – The DOGMA Approach, Jarrar and Meersman, http://www.jarrar.info/publications/JM08.v7.pdf

Tutorial on Ontological Engineering: “Part 3: Advanced course of ontological engineering”, Riichiro Mizoguchi, http://www.ei.sanken.osaka-u.ac.jp/pub/miz/Part3V3.pdf



Fragment: Symbolic vs Sub-symbolic AI

The symbolic-AI camp models knowledge as specific, explicitly-represented objective facts that get manipulated by formal, repeatable rules, and the sub-symbolic or connectionist camp is all about building systems that adapt, in hard-to-analyze ways, to perform actions and anticipate things in a way that seems to demonstrate knowledge but where the knowledge itself can’t easily be understood or extracted as a list of explicit facts or rules.

This stuff really works – Santa Barbara Innovators Program

Santa Barbara (wikipedia)

Last week I visited entrepreneurs and intrepreneurs attending the Santa Barbara Innovators Program.

I was inspired and energized by catching up with everyone and their progress. In this program we have a great cohort of five external startups, and three intrapreneur teams going though an intensive 12 weeks of business, customer and technology validation. At week five, everyone is starting to crank out new insights – alongside lots of techniques for customer empathy, and business modeling. It’s amazing what can be achieved with 10 customer interviews every week.

The Innovators Program combines customer development, design thinking, leadership training, and a focus on what it takes to succeed in b2b startups. It’s a Citrix Startup Accelerator initiative that also runs in Raleigh, Santa Clara and Bangalore. And you might well ask why we have intrapreneurs alongside entrepreneurs. This is some of the secret sauce – the cross fertilization and inspiration between our internal teams and the startups is simply amazing.

This stuff really works.

Citrix Startup Accelerator, and our Innovators Program are part of the Citrix Technology Office, and are focused on Open Innovation by working with startups.

Here’s a brief list of participating companies and Citrix teams – for more see the Innovators Program Santa Barbara page.

  • Encanta (Citrix): Customer interaction for mobile apps
  • Recapit (Citrix): App to capture knowledge from live meetings
  • Voitrix (Citrix): Improved lead information for incoming calls
  • 222LabsIoT for environmental and event monitoringCapyx: Cap Table Management
  • Caugnate: 3d extension of video conferencing for remote collaboration on field service work
  • Milo: Wearable alcohol sensor that actively monitors blood alcohol levels
  • Sanwood: Long life IoT sensor technology

All are interested talking with potential customers, so let me know (@michaelharries) if I can make an introduction.

R0010569On the way back to Silicon Valley from Santa Barbara, I visited Hearst Castle with my daughter. Hearst Castle is an amazing mansion built high in the hills of the Central Coast of California. From it, there’s an amazing view all the way down to the ocean, it seems that you can see everything. The Innovators Program is all about giving a similar clarity of vision to innovators, whether startups or intrapreneurs. 

On the dangers of “good” advice

“The aim of science is to seek the simplest explanation of complex facts. We are apt to fall into the error of thinking that the facts are simple because simplicity is the goal of our quest. The guiding motto in the life of every natural philosopher should be “Seek simplicity and distrust it.” – Alfred North Whitehead
from (http://scienceblogs.com/developingintelligence/2007/05/14/why-the-simplest-theory-is-alm/)

“Common sense is not so common.”
― Voltaire, A Pocket Philosophical Dictionary

“It is the obvious which is so difficult to see most of the time. People say ‘It’s as plain as the nose on your face.’ But how much of the nose on your face can you see, unless someone holds a mirror up to you?”
― Isaac Asimov, I, Robot

“Common sense is what tells us the earth is flat.”
― Stuart Chase, Language in Thought and Action

In short, to correctly interpret an aphorism you have to know the context, which isn’t present in the aphorism itself. Context is always vital.

The Future of Work – Amazing new technologies and Citrix Startup Accelerator (draft)

I recently attended the O’Reilly Solid Convention. it was very very cool, and right at the heart of current enthusiasm for ‘makers’, for IoT, and for hackery of all types. If you’re interested, go ahead and check out some of their videos. I’ll still be here.

One talk I found particularly inspiring introduced a new device designed to use emotive technology to very simply increase alertness or relaxation – the founder of doppel talked about some of the background to their work, exploring phenomena like the rubber hand illusion, and how this can be generalized with digital hands, and explored a range of doppelother cognitive illusions. The end result of their exploration was a very down to earth wearable device that provides a heartbeat like pulse that produces statistically significant changes in human physiology and performance. Unfortunately there is no video of this talk, but it is worth taking a look at their kickstarter.

I’m fascinated by these types of cognitive hacks, imagine how phenomena like this could enhance virtual reality, or make virtual meetings more productive. There’s amazing new innovation out there around giving new senses – such as seeing with the to2020 landscape docngue (now FDA approved), or explorations into adding a directional sense. The idea of using technology to make us stronger, faster, and more productive is introduced in the Citrix 2020 technology landscape document and is a one element of how our workplaces will change in the near future.

Another rapidly emerging area is the use of voice recognition. I’ve been using the Amazon Echo at home, and it’s amazing how quickly even the most technophobic of my family are willing to use this device. ‘Alexa, play <a random annoying teen pop artist> radio’. It’s fast, easy, and dramatically reduces barriers to use.

As of today, the types of commands provided natively by Echo are relatively simple. Yet even in challenging situations Echo can hear and understand requests so well that it feels like magic. Imagine having this technology in the workplace – in every meeting room. No more having to work out terrible user interface around different display options and more.

Check out a demo of this in action at Citrix Synergy 2015 with the Citrix Workplace Hub and Octoblu.

What about having meeting rooms that can identify who’s physically in a meeting, the discussion themes, if people are upset, or indeed foster better collaboration. Imagine having the Jarvis virtual assistant from ‘Iron Man’ at your work. Or, if thats not your scene, what else might be coming soon? How about instant hardware prototyping? What new IoT devices might make sense in the workplace? What if we could trial the hardware as simply and easily as software?

Future of workOne of the themes for Citrix Startup Accelerator is the ‘future of work’. The goal for this investment theme is to invest in first class startups, bringing new approaches and capabilities to these emerging themes in how our workplaces become more productive, and more human-centric. One example from our portfolio is WhoKnows. WhoKnows is an amazing company bringing very pragmatic improvements to the way we understand the rest of our team, and indeed everyone in the organization. I like to think of them as making the whole of a large company as simple to work with as a single workgroup. This is a huge challenge and one that existing approaches have not yet solved.

Do you have a great new approach to ‘The future of work’? Let me know.


Emerging Technologies conference (EmTech 2014)

emtech header 2

I had the good fortune to attend and speak at the 2014 MIT EmTech conference. I was particularly struck by Astro Teller and Yoky Matusoka talks. There were also awesome content on robotics, climate issues, hacking the mind, etc.

Here’s a link to videos of all presentations. http://www.technologyreview.com/emtech/14/video/

Astro Teller – Google [x]

  • GoogleX has a fluid structure and shared resources. Teller effectively functions as a board member for all projects.
  • Why is 10x the right measurement?
    • Get away from incremental thinking
    • “If just looking for a 10% improvement, engineers will start by improving the current <car>”
    • Radical requirement forces dropping assumptions
  • Philosophy of moonshot thinking
    • Enormous problem that can be named
    • Radical solution – if solution is clear/straightforward/well understood – nice, but not the ethos of their culture
    • Based on science and technology – and needing a breakthrough
  • Interesting – they have marketing in the group, but called by a different name … “Head of getting moonshots ready for the outside world”… can’t have “marketing” as it ‘scares off the innovators’ – [MH – Contrast this to ‘lean startup’ mindset, where marketing & customer focus, is front and center]
  • Projects are picked through a process that depends on Larry and Sergey intuition.
  • Google glass is not just a computer for your face – the real calling of wearables in general is to get out of your way – in exchange for being on your face needs to just work – no UI.
    • e.g. Teller’s life is ‘leveled up’ when glasses on and ‘leveled down’ when not – how to help digital world and physical world work without the schism. New Google glass leader is trained to be sensitive to getting technology out of the way using technology rather than thinking of technology as a benefit of itself alone.
  • Solve for X
    • More people should be doing Google X like things
    • Solve for X is a distilled version of this – find something that would make the world a radically better place – lots of incremental users
    • Originally thought there might be a pipeline problem – initially pushing hard for a conference to drink from firehose of new idea – but this turns out to not be the challenge. Actually want to do this, important for the world.
    • Structuring of molecules – hold same amount of gas at 1/4 of the pressure
    • Nick Negroponte – beam power to spaceship as it lifts off – capture and convert the heat
  • Project Loon – next year or so there will be a semi permanent circle of balloons in southern hemisphere

Michael Commentary:

** Very cool – sufficiently deep pockets to be able to solve the hard things, then worry about customers/market dynamics later.

Yoky Matsuoka – NEST (Google)

  • Started with a big problem – but one that can be embodied in a consumer product – easy to use and enriches your life
  • Background
    • Robots and neuroscience intersection – understand more of how human brain works using robotics tech
    • Use neuro understanding to build the right robots
    • See if robotic thecnoogy can help people with neuro problems
    • Created the center for Sensorimotor neural engineering
    • Relationship between device and human – tech can understand people – tech can do too much (people don’t learn) – too little not helpful — Ying Yang. Yokyworks engineering for the human experience
  • NEST
    • Solving big problems – solve problems that consumer apps …
    • 50% of domestic energy use is from heating and cooling
    • People are not good at doing this – potential to save at least 20% of the energy
    • 3 of 5 deaths happen in homes without working smoke alarms
    • Advanced technology in a beautiful package (otherwise known as the inner geek)
    • Saved over 2 billion kw hours – at least as compared with just running with fixed thermostat setting
    • Growing upsides from connectivity – collaborations
      • With Mercedes: The car knows when people are getting home
      • With Whirlpool – reduce noise by running only when people are away, run the fluff cycle when about to be home
    • Lessons
      • Continuing to learn about customers and deployment environment is critical
      • Initial assumption: if someone has purchased the nest, they will want to save power – hence a heavy focus on learning when to turn off the system.
      • However, for many buyers, the appeal is not energy saving, as much as beauty and adaptation to different household needs. Hence aggressive power saving was making them unhappy – NEST quickly adapted to a more refined approach.
      • Biggest surprise about how people actually use these devices – people touch the thermostat all the time. 1.6 touches per day.
    • Motivations: Matsuoka has an amazing background – the question came up for why she chose to do a thermostat. Turns out that some of her more forward thinking approaches did not gain market acceptance (robots for rehabilitation in the home, etc). NEST was the obvious and necessary next step as a way to gain acceptance for sophisticated devices in the home.

Michael Commentary:

** Nest is an amazing success story, but the main lesson here is around market dynamics, and the success Matsuoka had in identifying a necessary and viable innovation stepping stone.

Innovation and startups – a simple manifesto


Innovation is a dance between culture and technology. A matching of what’s possible with the magic of adoption of new things. It’s no good creating a technology only to discover that there’s nowhere for it to go. We have few chances to influence what will be, so those that we have should be treasured.

In 2011 I moved with my family from Australia to California to set up Citrix Startup Accelerator. The underlying idea is that we can be more effective at inventing the future, and seeing what’s coming next, not just by running research in house, but also by aligning with the ‘innovation machines’ of Silicon Valley and other global entrepreneurial communities.