Finite State Machines For Mr RIMP (Robot-In-My-Pocket) Workflows

Version 3 of @MrRIMP, my 3D-printed robot that fits in my pocket, is now a year old. Checkout our interview below!

In other words, I’ve been thinking about version 4 Mr. RIMP for a year! Long term I’d like Mr. R to satisfy the following requirements.

  • Nicer.
  • Funnier.
  • Smarter.

Ha! Seriously…

  • Capable of entertaining interactive behaviors (speech, animation, etc.)
  • Easy to create using commonly available Maker tools and components (cheap electromechanical parts, 3D printing, soldering, etc.)
  • Customizable at 3D-printed enclosure, hardware, and software behavior levels.
  • Leverage some sort of wearable/IoT connectivity (interactivity, provisioning, etc.)
  • Have some relevance to my interest in healthcare workflow technology (hence the next section of finite state machines in Python)

This blog post is really intended as just a place for me to archive some links relevant to the above requirements. (I find that I almost never go back to consult bookmarks. However I frequently go back to blog posts. And I like the idea someone else might stumble on this and then who knows what might happen!)

@wareFLO On Periscope!


Patient Narrative and Healthcare Workflow: Story Informatics!

I have a particular interest in the relationship between stories and workflow. On one hand I’m ABD (All But Dissertation, that story here…) in computational linguistics (the science behind the engineering of natural language processing). On the other hand, I’ve spent my professional career focusing on various aspects of workflow technology in healthcare. Often, when dealing with nontechnical users, I substitute the word “story” for the word “workflow.” Workflow sounds scary (or at least boring). Story sounds friendly (or at least interesting). If I were to coin a phrase, Story Informatics comes to mind!

Many of my projects during my graduate degree in computational linguistics involved analyzing various kinds of narratives: Pilot’s conversing with co-pilots, physicians thinking out loud during problem solving, depressed patients ruminating about life, to name just a few areas. The workflows patients experience give rise to natural language descriptions, from patients, of their experiences interacting with those workflows. Wouldn’t it be cool is there was some way to convert those natural language descriptions into models of workflow? In fact, research leads toward this goal.

We don’t have to wait to use patient narrative to create workflow systems. Right now analysts create journey maps from customer descriptions of their experiences. My favorite phrase here is “empathic workflow.” As in design thinking, where empathy of users is so important, empathy is also so important when designing workflows with workflow technology.

I am looking forward to today’s (every Friday at noon EST!) #HITsm tweetchat Using patient narrative to explore gaps in health data interoperability. Although, of course, I always substitute “workflow” for “data” (try it!): Using patient narrative to explore gaps in health workflow interoperability (AKA pragmatic interoperability).

@wareFLO On Periscope!


Deep Learning, Pattern Recognition, and Healthcare Workflow and Interoperability

I was actually present, in a down-the-hall-occasional-beer-together sort of way, at the beginning baby steps of deep learning, a branch of machine learning, a branch of artificial intelligence (one of my graduate degrees). I was taking a graduate course in connectionist models and neural networks at CMU. Some students worked on neural networks with hidden layers. Several worked on recurrent neural networks, networks in which the output of the network feeds back into the network. And some of this work laid the foundations for today’s deep learning success.


Hidden layers are necessary to learn certain abstract concepts (such as Exclusive-OR). Recurrence is necessary to learn certain patterns of behavior over time (such as sentence structure, or, I speculate, workflows!). I even built neural network models of mental illness for one of my research projects. (Computer Modeling of Adaptive Depression and Asymmetric Hemispheric Processing, 1996) Some aspects of neural network optimization (that’s what learning is, in this model) resemble techniques I learned about in my industrial engineering courses.

Ultimately I moved off into other academic directions, ending up with an MS in Intelligent Systems. But I’ve kept my eye on neural network research, especially as it emerged into commercial use. More recently I’ve seen reconvergence with my interest in healthcare workflow and workflow technologies. In particular, machine learning methods are increasingly used to recognize context and predict what users need or wish to do next. Think Google Now on Android. Machine learning systems, like other data pipeline-oriented systems, resemble, in some respects, workflow management system architectures.


With that history in mind, I recently downloaded the free and open source TensorFlow (Flow!) deep learning software from Google. I’m going through the tutorials and seeing a lot of familiar ideas, except they are no longer half-baked, like they were back when I was a graduate student. I have not decided what data sets make sense for me to mess with (got one?) so I’ve been poking at the 2016 HIT100 data set made available by @ShimCode (thank you very much). What would be interesting to recognize or predict? Who will be nominated? Whether they will make it into the top 100? Their likely eventual rank? What would be the inputs? Who nominated who? Characteristics of their followers and/or followers? The actual contents of the tweets? Love to hear some suggestions!

On to the HITsm tweetchat questions!

Topic 1: What jobs will be affected most by AI and Deep Learning in Healthcare? #HITsm

Immediately? Visual pattern and object recognition: dermatology, radiology

Topic 2: What types of initial diagnoses would be helpful to be made by AI algorithms #HITsm

Just like certain kinds of ECG diagnosis are almost baked into heart monitors, I think you’ll see cheap apps and hardware for skin and certain imaging tasks (ultrasound comes to mind, less risk of radiation).

Topic 3: What healthcare decisions would you be comfortable being made by AI, if a human reviewed, life or death abnormalities? #HITsm

You’ve heard of shared decision making? Between patient and clinician? Well, consider shared decision-making among patient, clinician, and intelligent systems. The classic model of shared decision making (back even before it was called that) was the Vulcan mind-meld between clinician knowledge and experience and patient values and goals (Healthcare Trade-offs, Shared Decision Making, Vulcan Mind-melds, and a Marriage Metaphor). Well, if clinical knowledge can be artificially machine learned from experience, we are now looking at a mind-meld among patient, clinician, and machine.


Figuring out and allocating competencies and responsibilities? That involves my favorite topic, workflow, as I’ve written about here.

Topic 4: If an AI could be trained to recognize body movements and facial cues, would you be willing to utilize automated visits with telemedicine followup? #HITsm

Patient monitoring is largest wearable/IoT growth area, and in principle includes gesture and facial expression recognition. Recognizing categories of human behavior can be more than just visual, it can include other sensor data as well. Increasingly, these wearable/IoT/machine learning systems will learn to recognize more-and-more abstract human states, such as emotions (affective computing) and goal, plan, and task/activity status: workflow! (Though some might say “life-flow”).

Topic 5: What type of pattern recognition could be used to enhance interoperability in healthcare? #HITsm

I frequently promote what I call pragmatic, or workflow, interoperability. Syntactic and semantic interoperability are about moving data and shared meaning. Pragmatic interoperability is about how actionable data is used. An important aspect of pragmatic workflow interoperability is goal and plan recognition. If I observe you doing something, and I recognize what you are trying to do, and I can help, that is a form of interpersonal interoperability. Something similar exists between any two coordinating entities, be they people, robots, or organizations. Deep learning can play an important role in recognizing goals and plans and then triggering helpful actions and workflows.

I’ll see you at the #HITsm tweetchat! [#HITsm chat 7.15.16] Deep learning in healthcare.

@wareFLO On Periscope!


Thank You For Mentioning Workflow When You Nominated Me to #HIT100


I’ve not always been enamored of the annual #HIT100 nomination tweet fest. I came round when I realized how useful it can be increasing awareness of the importance of healthcare workflow and workflow technology. I literally looked through thousands of followees and followers for my nominations. I added my own hash tag, #PragmaticInteroperability (more info here) to each tweet. This year I’ve enjoyed the #HIT100 process immensely.

I’d like specifically thank each of the following health IT twitterati.

(If you mentioned workflow in your nomination, but perhaps used an alternative spelling 🙂 let me know so I can add you to my treasured archive.)

Thank you Sean! I’ve enjoyed your blabs. Your combined interests of social work and workflow (“social workflow”) is an interesting addition to the health workflow IT conversation. I look forward to seeing it evolve further.

Thank you Nick! Our conversation about healthcare workflow, natural language processing, and workflow technology, during your Internet radio talk show, was one of my early opportunities to raise healthcare workflow consciousness.

Thank you Colin! You have a generous spirit. You make every one of us, who have our own narrow individual obsession, feel like you are channeling us at conferences, in blog posts, during tweet chats, blabs, and periscopes.

Thank you Jim! Very cool to have met you, a radiologist, several years ago at the Healthcare Process Improvement Conference in Orlando. And also very cool how process improvement methods are finally coming together and merging with health IT and workflow tech!

Thank you Sarah! An Batman thanks you. (To the Workflow-mobile!)

Thank you Dirk! And thank you for sharing what gets your goat during One-Minute Hatcam Interview!

Thank you Joe! And thank you for letting me use your handsome mug on my Twitter avatar during AHIPinstitute!

Thank you AdvancedMD! Thank you especially for noting the workflow+interoperability connection. Not only is interoperability necessary for great workflow, great workflow technology can help may true healthcare interoperability a reality.

Thank you Faisal! And thank you for following back when you were the 6th person I followed on Twitter!

Thank you Matt! You had me at ‘workflow’!

Thank you Janet! And thank for attending so many of my Periscopes!!

Thank you Peggy! We, JETS! The @HealthITdog and I, hope to bump into you at some health IT event here in Columbus, Ohio, sometime!

Thank you Linda! And for letting me interview you, at length, when you were #HIT1 (out of #HIT100).

Thank you Manish! Love your healthcare workflow history and motivation, which I published on this blog: How Does Manish Sharma Know So Much About Healthcare Workflow Technology? His Answer!

Thank you FormFast! I recall as if it was only yesterday, our co-hosted webinar, The Power of Process (almost 2000 views on Youtube!).

Thank you Vince! The first shall be last. You’re the first person I followed on Twitter. Thank you for being an excellent health IT & healthcare workflow discussant!

@wareFLO On Periscope!


Professionalism, Etiquette, and Workflow in Health IT Social Media

Believe it or not, “professionalism” has a lot of definitions. If you are inside a profession, such as medicine or law, then you likely have a lengthy and specific set of guidelines, or instructions, about how to maintain your professionalism. If you are a member of the public, dealing with professionals, you have a shorter, more informal, sense of what it means for them to act professionally. And if you are a sociologist, you have yet another (set of internally debated) definitions.

I’ll start with my elective course in medical sociology during medical school. It was taught by Odin Anderson, who has been called the father of medical sociology. To the sociological eye, professions serve to self-police (maintain standards) and maximize economic position.

Many groups within society aspire to become professions. The medical and legal professions are considered the gold standards. Other groups, such as engineering and teaching, are partially professionalized. Most would agree that the social media “profession” has a long way to go before it becomes a true profession. Indeed, some would probably argue that social media work shouldn’t become too professional, as that might kill the goose that lays the golden egg, so to speak. An important element of social media is it’s impromptu and edgy nature. “Edginess” tends to disappear when rules and norms arrive. I suspect we’ll see a dynamic and evolving mix of the two extremes.

In contrast to the sociological view, medicine describes professionalism quite differently.

“The professionalism charter defined three fundamental principles of professionalism:

  • The primacy of patient welfare: This principle focuses on altruism, trust, and patient interest. The charter states: ‘Market forces, societal pressures, and administrative exigencies must not compromise this principle’.
  • Patient autonomy. This principle incorporates honesty with patients and the need to educate and empower patients to make appropriate medical decisions.
  • Social justice. This principle addresses physicians’ societal contract and distributive justice—that is, considering the available resources and the needs of all patients while taking care of an individual patient.”

By the way, the Jordan Cohen mentioned below was my advisor during medical school!

Then there are more general treatments of professionalism.

Due to all these difference senses of what it means to be a profession, to act professionally, and with professionalism, I actually prefer old-fashioned words, such as manners or etiquette.

From Ethics in Computing:

“Etiquette refers to a code of behavior, a set of norms of correct conduct expected by a society, group, or social class. It is a generally expected social behavior. These rules of the code or the set of norms are usually unwritten, but aspect of these relay reflect an underlying moral code. Manners are unenforced standards of conduct or cultural norms that show that an individual is “refined” and “cultured” with a society or group. These norms codify human behavior, manners, just like morality, have no formal system for punishing transgressions other than social disapproval.”

Many books about etiquette and protocol are, essentially, filled with workflows, such as exactly how to greet someone, take leave, and make introductions. These workflows arise and evolve over time. They are phenotypes reflect practical environmental constraints, human purposes and goals, and personal moral codes and organizational values (the corresponding genotypes).

Now, how does all of this apply to professionalism in social media?

I’m sure you can think of examples in the #HITsm, #HCLDR, #HIT100 and related healthcare social media communities during which generally accepted rules and norms were followed scrupulously. And you can also think of episodes when rules and norms were not followed, and then there were direct and indirect expressions of social disapproval. I suspect these rules and norms have evolved immensely from the early days of social media, and will continue to do so.

@wareFLO On Periscope!


Can Health IT Social Media Help Health IT Plus BPM Reach Critical Mass?

You know what “critical mass” is, right? It has two definitions, one from physics and the other outside of physics. In physics critical mass is the minimum amount of material to cause a nuclear chain reaction (in other words, an explosion). Outside of physical critical mass is the minimum amount of something to start or maintain a venture.

I’ve been trying to drive two areas of technology together for years… For the last half decade I’ve used social technology to do this. I hope, eventually, to achieve the minimum interaction between medical informatics and workflow informatics, and between the health IT and business process management industries, to trigger a chain reaction, a chain reaction of ideas and conversations and technologies and successes that will feed on itself.

That will lead to widespread automatic, transparent, flexible, and systematically improvable healthcare workflow systems.

We are very, very close.

P.S. Like my animated GIF?

@wareFLO On Periscope!


How Does Manish Sharma Know So Much About Healthcare Workflow Technology? His Answer!

You [@MSharmas, @HCITexpert] know what I’m talking about when it comes workflow tech and BPM, where did you get that experience/knowledge?

Manish’s emailed answer:


Before starting out to be an entrepreneur in 2012, I was leading teams to develop a cloud based EMR solution for the company I worked for, wherein we wanted to incorporate the following in our core frameworks:

  • Workflow Engine,
  • Rules Engine,
  • Alerts Engine
  • Forms Designer

In the process of selecting the workflow engine in our Product, we had done an in-depth analysis of the various technologies available to enable tweaking of workflows. We looked at open source solutions and proprietary and had worked with the teams from Skelta towards PoCs for billing, insurance, inventory, pharmacy and CPOE workflows.

This is when I was able to learn about the benefits of having a workflow and rules engine in our product.

And hence I understand the concepts you talk about and the need for incorporating workflow tech in Healthcare, simply because I believe every Healthcare Facility is unique and most times I have found companies sell the product, but deliver a “Service” to their customers by trying to do a GAP analysis to come up with the actual solution required by the Client.

If BPM, workflow tech was there, your classic question during the “Demo” would have come to help a lot of customers in understanding what is the meaning of workflow technology.

Not yet learnt all of it, but its a work-in-progress 🙂

Thanks a lot for your question …

@wareFLO On Periscope!


Blab: A New Way To Demo Health IT Software!

Believe it or not. That Hollywood Squares-esque social video platform, Blab, mostly populated by people talking about how great Blab is, is a really great way to demo health IT software! @dflee30 used the Blab Chrome extension…

…and gave about an hour demo of the ZenPRM collaboration and provider resource management system. Well, it would have been an hour, except I kept interrupting and asking questions. Which Don answered impeccably.

Couple of things to note. The live blab looked way better than either the replay or the youtube version. In both of the latter, the screencast is squeezed into a corner quarter of the video. But in the original blab, it stretches gloriously and crisply across about the top three fifths of the screen. I’m sure best practices will emerge, but in this case what seemed to work was lowering the screen resolution and bumping up the font size on the sending end, and playing around with the browser size and aspect ratio on the receiving end.

I am already looking forward to many demos of healthcare workflow software!

@wareFLO On Periscope!