Understanding Bad Behavior

Recently, The Medicine Maker has examined how gaming technology, including actual video games and VR headsets, can be used as treatments for certain patients, but could gaming have an impact on the pharma industry in other ways? As part of her PhD, Heather Campbell used video games to study human behavior when making decisions about the quality production of drugs (1, 2). After all, a company can have the perfect machinery and perfect chemistry – but, in manual processes, human behavior can change everything. Campbell’s supervisor, Robert Lodder, a Professor in the Pharmaceutical Sciences Department of the University of Kentucky College of Pharmacy, explains, “Games can help predict what people will do. If you want to know how people are going to break your system, try playing some serious games.”

So can games really tell us how people might break a drug quality system?

From analytical chemistry to computer

With Robert Lodder

I received  my PhD in analytical chemistry from Indiana University Bloomington, but I really enjoyed combining computers and chemistry. Over time, I became good at using computers, and I even won first prize in an international IBM supercomputing competition in the life science division. I’ve spun out a few drug and medical device companies from academia, and I’ve also worked on projects with the likes of DARPA and the Department of Homeland Security. For example, Homeland Security had an analytical method they wanted to test to look for chemical weapons being placed in the US food supply. We came up with a gaming approach that used simulated attacks to test their method (in some cases proving the analytical method wouldn’t work).

Such gaming approaches can be used in many different industries – and I’ve been working with Heather to show how gaming can be used in drug quality.

The University of Kentucky Hospital is the only hospital in the US that has a regular, full-time program analyzing incoming injectable drugs before they are used. The program was launched in late 2019 and testing began during the pandemic. The program has found a number of potential problems that have already been reported to the FDA through MedWatch and Citizen’s Petitions as well as published (see Drugs With Reported Problems below). Pharma companies must adhere to GMP and GLP – and there are regulatory inspections to enforce this. In her research, however, Heather read that Valisure (a company in Connecticut that tests oral dosage forms) found around 10 percent of drugs have something wrong with them.

Drug companies often buy their ingredients from China or India. Companies will look at the label, the chain of custody, and then test the ingredient, too, before using it. On the other hand, when a pharmacy receives a drug, they look at the label and the chain of custody only – and then use the drug. If we want to eliminate the last 5 to 10 percent of problems, pharmacies need to test drugs on-site before they are used. And if the drug isn’t right, they need to send it back to the manufacturer and tell them to try again! We should not assume that everything is equivalent. Generics, for example, are all assumed to be the same as the brand product. During the pandemic, the FDA halted foreign and even many domestic inspections. Even in normal times, there are still things an FDA inspection won’t always catch (like the employee running down the hallway with bags of shredded documents as described in Catherine Eban’s book, Bottle of Lies). Many inspections are announced beforehand so companies will be able to prepare. In other words, there is always a danger of low quality drugs entering pharmacy supply chains. Counterfeit products are also an issue.

Even at our hospital, we don’t have the manpower, space, or equipment to test everything when it arrives so we prioritize drugs for testing. Some of the considerations include the manufacturer’s Form 483s from the FDA, whether the company is the sole source of a drug, how much a drug costs and how profitable it is to counterfeit, among other things. Typically, however, that information can be months or even years old. Current prediction models tend not to fare well when faced with unexpected events – like the COVID-19 pandemic. They are also not always good at predicting human behavior. 

Heather’s work looked at how a serious gaming approach could simulate how humans react in a pharmaceutical environment. If we can understand how humans behave – and the quality shortcuts they may take in the pursuit of profit – it may help us to understand where the biggest risks lie and help to improve current risk prediction models.

A game to test ethics

With Heather Campbell

My undergraduate degree is in mechanical engineering and then I opted for a PhD in pharmaceutical sciences. At first, I was in a formulation lab focusing on spray drying and traditional pharma stuff, but then I started working with Robert Lodder. He suggested that I do something with games for my project. “Games?” I thought to myself. “What on earth can we do with games?”

I initially thought about a way to simulate the spray dryer, but the industry already has that kind of simulation technology for manufacturing. Games are really about humans and can allow you to model human decisions, so I thought about it some more – and how human behavior and decisions can affect drug quality.  

I programmed a Python shell and combined it with an off-the-shelf video game: Big Pharma. Big Pharma is a game released by Twice Circled where players build and run a pharmaceutical factory. They have to build processing lines, choose different ingredients and make different APIs in the right concentrations. There are also different aspects to the game, such as marketing your drugs. You can even make the API concentration lower than it should be (and sell it to your customers!). And yes, you can commit various types of management fraud too, including concealing clinical trial results. It was something I could get started with quickly rather than inventing a new game from scratch.

The custom Python shell kicks in every now and then during gameplay by stopping the game and adding different activities – many of which are SOP related. For example, a pop up message might say, it’s time to inspect the equipment, or perform cleaning. It may also flag other problems, such as rust particles in products or record keeping issues. Players can choose to ignore the message, take action, or, in some cases, fire the person who raised the problem. Taking action to remedy a problem or conduct maintenance or inspection costs money – just as it would in real life. There were also options for workers to leave early or to sign off on cleaning without actually doing it.

And herein lies the problem – unethical practices can be rewarded with monetary gain (until you get caught!). Doing things properly and adhering to quality standards doesn’t always come with visible rewards.

We had pharmacy students play the game – and most of them were very good and ethical even though the player with the highest profit won the largest cash prize! Obviously they are trained to do things properly (and I think they were aware they were doing this in front of a professor!). But there were “cheaters” who made some very unethical decisions. And these players did a lot better in terms of profit than our ethical folks. In many cases, they were selling sub-potent API (active pharmaceutical ingredient) and engaging in substantial price hiking.

We had students play through three different game variations. In the first game, the aim was to make the most profit. In the second game, players had to gain the most quality points. The third game asked players to maximize profit and quality. Those who engaged in more unethical practices had the largest profit, but rewarding quality with quality points did help improve the behavior.

I have also been working on a second, more detailed game, which more closely resembles a real 3-D pharmaceutical production environment. You have to scan your badge to enter the lab and will have to put on cleanroom gowns in classified areas. You can look at the equipment, inspect it, adjust valves, among other conventional procedures. There will be scenarios where the equipment is filthy – and you will need to ask if you have enough hands to clean it. It’s about replicating scenarios that may come up in real life. This game will provide a deep dive into the shortcuts people might take. Do you rush your cleaning and go home early? Or do you stay late and clean the tank properly? 

What’s the score?
 

People can (and will) take far more risks in a game because there are no real world consequences. When real patients and lives are involved, we all want to believe that manufacturers will take the highest care. But there is no getting away from the fact that the results from Campbell and Lodder’s game are familiar. Sub-potent APIs are a relatively common occurrence in the industry – whether through genuine manufacturing errors, operator error, or a more devious management strategy to save money. In 2020, for example, a drug quality study at the University of Kentucky found impurities and low levels of API in the injectable diuretic, acetazolamide (3). According to Campbell’s research, two of the companies selling the subpotent versions of the medicine were also selling substantially higher compared with another version of the drug, which remained on the market without quality issues. Interestingly, the same scenario was seen in the gaming situation, with many players choosing to sell their sub-optimal APIs at significantly hiked prices. Campbell also notes other studies that have observed higher prices for low quality drugs (4). 

The conclusion may be a sobering thought for pharma manufacturers: the drive for profit can lead to problems, especially when companies prioritize profit over patient safety. Currently, all generics are assumed to be equivalent and companies compete only on the basis of price.  Campbell and Lodder suggest that a system that rewards quality could have real-world benefits. Testing such a system in reality would be expensive and time consuming, but here we have a virtual gaming approach that shows how it could work. Pharma companies are still run by humans – and the simple truth is that humans need to be motivated. 

And even if we can’t adequately reward quality or identify all potential culprits, the results from gaming scenarios could at least lead to better risk models – either for pharmaceutical companies themselves or regulators or labs performing limited independent analysis, like the Drug Quality Study (DQS) at the University of Kentucky Hospital.

What lies ahead? Campbell is moving on to start a career in drug quality. Meanwhile, Lodder is thinking about running a larger experiment using gaming to explore drug quality using a gaming tournament approach. He also says that people shouldn’t be quick to dismiss the work as a gimmick. He says, “After all, flight simulators have been used for years to successfully train pilots. What about simulators to teach people to make better drugs?”

You can read Campbell and Lodder’s paper in full here or Campbell’s full dissertation here.