The Corruption of Trust in Oregon’s Cannabis Labs

Without action, public and industry trust in cannabis laboratories will continue to erode.

Jamie Toth, The Somewhat Cyclops

04 Feb 2022 • 16 min read

Image Source: Author

I enjoy writing about cannabis science because it’s an exciting time for cannabis. I like helping other people to understand what they are seeking from their cannabis experience — and figuring out how to get it. I try to process what science has to say about how cannabis impacts us and demystify it to make it more understandable for people seeking its benefits.

When I first started exploring the story about laboratories in Oregon in 2019, I thought that the story was whether there are some rogue labs in Oregon. Someone in the industry advised me to stop writing about it. If I was a journalist, that would have told me that not only did I have a story, but I had a good one.

I’m not a journalist.

Two years later when I heard people in the industry say that it was still an issue, I decided that not only was there was a story,- I wanted to know what it was. To chase down that story, I pulled the list of labs across Oregon and planned out a round robin by calling every lab on the list. It was an extraordinary and humbling experience — there are a vast number of dedicated scientists who want to produce the best number possible. I talked with people who have a deep abiding love of science and cannabis and want to do best by them both. They are inspirational people whose work is threatened by the power of strong economic forces.

In some cases, they reached out to me from to not only tell me that they suspected their lab was fixing results, but how. Some laboratories shared emails where they had retested samples from shelves, and sent the tests to the state. Still others offered advice on pertinent studies.

Hearing their stories and fears and getting their advice about how to proceed made me understand quite a bit more about the cannabis industry in general, in addition to the limits and challenges of those working in cannabis science.

It was their advice I used to help formulate the round robin plan.

But the biggest issue? Funding.

I couldn’t crowdsource — I would be using part of the funds to acquire samples and purchase cannabis-related services. Since cannabis is federally illegal, collecting money with the intent to buy it is also federally illegal. I moved to a legal state and became a medical patient to avoid breaking laws.

The only legal way to move forward with the round robin was with funding from within the industry itself. To perform a true blind test would require being able to front all of the lab fees — over $10,000. Then, the pure logistics of trying to get samples, homogenize them, and get them to the labs — all without the labs knowing it was related to my ‘blind’ experiment?

It was too much financially and logistically for a writer to bear. Honestly, it’s also outside of my scope — If someone wanted to discredit the work, all they would have to do is discredit me. I don’t have scientific training, or letters after my name. As someone told me — I wouldn’t be the first to try it and I wouldn’t be the last.

When I talked with my State Representative, John Lively, about funding issues and logistic issues, he advised me that the only legal way forward for me was funding from within the cannabis industry.

The only people interested in funding? Cannabis labs whose business was under threat. Which meant I had to ask the labs to trust each other.

Spoiler Alert: In many cases, they don’t.

While I had initial consensus from the labs, once one stepped up to the forefront to help, interest in the project evaporated because the conflict of interest was clear.

There will still be a round robin, but on a smaller scale. The results will still be relevant, but still dismiss-able by those who won’t like the results.

What the Data said . . . and What it Couldn’t.

As a data architect, more of my hopes hinged on the public records request. I performed an analysis of the 250,000 records for THC tests that the state of Oregon keeps in its Cannabis Tracking System. I quickly received four years of de-identified data in quick time, and I’m grateful to TJ Sheehy and his team who made that happen.

The records request showed not only issues with the labs, but also issues with data collection and governance ( those are always to be expected in data sets like this). But even though it was a perfectly formatted imperfect data file, it still had a lot of relevant information and told quite a story.

In a prior story, I wrote about research into issues with labs in Washington and Nevada. That research gave evidence that there were issues in the labs by examining a frequency analysis and noting the ‘bump’ that occurs around 20% THC, which is a significant number for marketing.

Here are the graphs from the Washington and Nevada Study.

Image Source: https://jcannabisresearch.biomedcentral.com/articles/10.1186/s42238-021-00064-2

Using my 20 year background in health care data analytics ( I’m even published!), I performed the same graphing on Oregon’s test results. Unlike other areas — I have expertise in this one.

Here is a graph of Oregon’s 2021 THC Potency Results for product entered as ‘Buds’ in METRC.

Here is that same data set, with three labs eliminated. How did I pick the labs? By looking for labs that had abnormal behavior compared to the above curve.

I understand why people would say ‘that doesn’t really say anything.’

To me it says a lot.

I heard how many thought that perhaps cannabis results wouldn’t conform to a normal curve — and yet — with just three labs eliminated, the curve has started to look far more normal. It doesn’t peak at 20, nor should it — the average THC concentration in Oregon is actually at 23.3 for the year of 2021. Which is exactly where this data set peaks.

Isn’t data neat?

What’s the Story, Morning Glory?

Once I allowed myself to step back a moment, I was able to see deeper so I could process what I had experienced.

Scores of phone calls reinforced that many in the cannabis industry believe that some rogue labs are fixing the THC results . These people are often so certain that they will name names — and claim the state is unable or unwilling to deal with the problem. Despite the fact that many in the industry believe there are rogue labs, they still engage with them. Farmers still get their cannabis tested there to reap the economic benefits. Dispensaries still stock product tested by them, while admitting they don’t believe the posted numbers. Labs still subcontract for them.

I thought this was a story about whether there are ‘bad actor’ among cannabis laboratories in Oregon, but I was wrong.

This is a story about the how there is an industry-wide erosion in trust in the ability of cannabis labs to deliver accurate scientific results in the face of extreme economic pressure.

Ever the stickler, I wanted undeniable proof.

The issue is — if someone’s job hinges on not believing you, it doesn’t matter what proof you bring to the table. I was even told as much. For those people — no proof someone like me could provide would ever be good enough. For those people, nothing is undeniable.

The Open Secret.

This is the part that hurts the most for a variety of reasons, but I’ll stick to the less personal ones (and even those are personal).

I’ve spent so much time evangelizing that consumers need to check lab reports and lab information that it hurts to find out there’s such a serious issue. It’s disheartening to know how many budtenders, shop owners, farmers, wholesalers, and yes, lab technicians — that know this is going on. What’s worse is seeing how little the cannabis industry has tried to educate the consumer to try and take the pressure off of the labs.

The ‘open secret’ situation around some rogue Oregon labs fixing THC potency is a textbook example of corruption. An authority (laboratory) manipulating results to their economic benefit — right out in the open.

Open secrets aren’t well-kept. All you have to do to be in on this one is call up a dispensary with a high tester with a ridiculous low price — and ask them who tested it. Everyone will tell you about it, but no one will want you to print their name.

I’ve heard it said that the standard you walk past is the standard that you accept. Applied to this situation, it’s clear that regardless of their willingness to talk about their suspicions, many in the industry are fine with the status quo.

Don’t believe me? Look at who offers consumer education and who doesn’t. You can determine what companies value by what they spend resources on.

All you have to do with this is follow the money.

Some labs, when feeling defensive, asked me what I was doing to ensure people were being educated about cannabis. One ranted at me for over two and a half hours about how my idea was stupid, and I was an idiot for pursuing it. Another told me that writing about the cannabis lab issue in Oregon was not only a waste of time but would ‘ruin me.’ They continued to tell me how better people had done the story (referring to a piece in a local paper) and had been ‘shut down.’

But that’s not where the accusations stopped.

‘It’s the Consumer’s Fault!’

The number one comment I heard from people across the industry is that it’s the consumer’s fault. It’s the consumers seeking THC, it’s the only thing they want, it’s the only way they order. According to some industry experts, the entire situation is due to the ignorance of the consumer.

It doesn’t take much research to see which cannabis brands have invested in consumer education. An easy way to find which dispensaries, farms, and labs invest in consumer education is by a simple web search. After COVID-19 there’s no longer an excuse for cannabis brands to eschew the internet. It’s surprising how few have established websites, engage with consumers about their services and credentials, or give material and links to valid research stated in simple terms.

The industry that claims it dangles on a precipice because of consumer ignorance hasn’t made major strides in educating consumers.

Letter of the (Labeling) Law

It’s true that many might not understand the impact of what the labeling rules actually say. It’s easy to admit that the accusation that it’s the consumer’s inability to understand what the label is telling them might have some validity. It’s also easy to see that the misconceptions of what cannabis labels say can’t cover the scope of the issue.

Redacted Oregon Cannabis Flower Label. Image Source: Author

One of the big points made in the General Requirements section is that “Packages cannot contain untruthful or misleading statements.” As someone who used to work in a field full of stringent regulatory requirements (healthcare), I read that to be a blanket statement, so that if any statement on the label is willfully falsified, that it’s an actionable infraction.

There’s also a delightful nuance in that the rules that states the THC and CBD for usable marijuana as dry weight — but this doesn’t mean just a bit dry. This means cracker dry — drier than you’d ever want to buy it. That means that the THC on the label will seem slightly higher than it should be, because the cannabis inside (hopefully) still has a bit of moisture (less than 15%).

Let’s talk about the star of the show and the center of this drama: THC.

The most important thing to understand is that what you’re seeing on a label for “THC” is, even in the best of cases, an estimate. Even if the number on the label was 100% accurate, the lab can’t have actually tested the bud you’re about to enjoy. Part of how labs derive that number is by using sampling techniques performed on batches of material. ORELAP accredits the sampling technique of each lab, and lab personnel must follow those documented and accredited procedures. Currently those batches are 15 pounds, but there is talk of increasing the batch size to 50 pounds.

I don’t have confidence that increasing the batch size of 50 pounds is in the interest of consumers. In a batch of 50 pounds, it’s rather meaningless, unless the Relative Standard Deviation, a statistical metric that describes how much variance there is in results, was required in addition to THC.

To complicate matters more, the rules for labeling state that the THC number reported should be the potency at the time of consumption — not at the time of testing.

Don’t have enough complication yet? The potency only expected to be within 20% of the value on the label. This means it’s acceptable, by law, that an 18% cannabis carry a label for 21% and be considered accurate. That doesn’t seem like much, but there are large economic incentives noted in the market for cannabis that is over 20%.

But if you bought something that said it was 91 octane, and it was 87 — would you be angry? Would you trust the vendor again?

There Ought to Be a Law

Cannabis Laboratories in Oregon are licensed by the OLCC but have their processes, procedures, and equipment accredited by ORELAP. When I wrote to the state about my concerns around cannabis potency testing, Jonathan Modie, Lead Communications Officer, explained, “The rules around testing potency on cannabis try to address the issues you have raised. The OMMP, which resides in Oregon Health Authority, has the authority for setting testing rules for cannabis. The Oregon Environmental Laboratory Accreditation Program (ORELAP), also in OHA, has the authority to oversee testing laboratories and accredit them to national standards that are based on ISO standards but specifically tailored to environmental testing laboratories. Oregon does not have a reference lab which would be able to test products and act as independent third party to assure results from labs are accurate.”

Unfortunately, many laboratories believe that these national standards fall short in giving guidance and standards criteria appropriate and specific to cannabis, and without an established reference laboratory or reference mechanism, there’s no consistent way for the state to ensure consistency in testing across the laboratories.

Viruses and Water Wars

Modie’s email went on to explain, “ORELAP performs site visits to laboratories to ensure they are performing their testing according to set standards. Cannabis testing laboratories must go through a round of blind third-party proficiency testing for potency every 6 months to ensure they are utilizing methods that produce accurate results. If a laboratory does not maintain a record of acceptable performance in at least two out of the past three proficiency tests, the laboratory’s accreditation is suspended for that testing. A laboratory’s suspension is lifted when they can show acceptable performance.”

I had already heard from laboratories that there had been gaps in the onsite audits due to the logistic complications brought on by the pandemic. Many labs also pointed out flaws in the six month proficiency testing system that could be exploited if a lab was a ‘bad actor.’ Some of these flaws included how easy it was to identify the samples for proficiency testing.

A raging pandemic that is changing how the state is able to audit isn’t the only thing that is on Oregon’s plate. Since the legalization of industrial hemp t here have been serious issues in Southern Oregon around illegal grows, especially on land that had once grown hemp, before the market bottomed out . News reports now include stories about massive drug busts, murders, and police who feel outmanned. Residents live in fear — not just of armed men in unmarked pick up trucks full of cannabis speeding down the roads, but of creeks run dry far earlier than usual, and run off from the farms polluting their streams and creeks.

Regardless of their growing war chest of licensing fees and tax revenues, all of this is a strain on OLCC’s finite resources.

There is a public comment hearing on the 16 of February to review new proposed testing rules , and I’ve asked to share what I’ve found over the course of my adventure. Reviewing the proposed expansion of rules, I am still concerned that they are not sufficient to protect from some of the issues I’ve outlined here — and I don’t see any specifics calling out mining the data available in the CTS to support audit efforts.

But the fact of the matter is — I’m a writer.

The stories I have to tell about cannabis lab testing are other people’s stories.

The people who should be speaking at this hearing are the dispensaries, laboratories, and farmers who have not only expressed their sense of damage that the situation has done to their business to me, but the ideas they have to fix it.

As in most things, people have far more complaints than solutions.

What Can Be Done — and Who Would Do It?

It’s an overwhelming amount of information, I know, but the most important thing to think about is how can a situation like this be changed — before trust is completed eroded and honest laboratories go bankrupt?

Oregon could increase data stewardship efforts so they could use METRC data more effectively

It was evident from looking at the data that there were issues. There were lab results that were far too high for flower (I can’t imagine an 88% flower), and others that were far too low for concentrates. It’s understandable that full data checks could not be performed in METRC’s front end. A strong data governance effort that focused on stewardship at the laboratories that included reports to highlight issues would increase data accuracy within METRC.

Some of the pushback I received when I originally vetted the information I had gleaned from the data set that was METRC was the concept of garbage in, garbage out — and data governance projects would be an effective way to address those concerns. Since METRC requires coordination of data entry rules across the state and across organizations, without comprehensive training with centralized messaging and monitoring, the data will continue to be troubled.

Oregon Could Change Flower Labeling Rules to Include More Available Information

The rule could be changed to include the RSD in addition to the THC — especially as the new rules acknowledge its importance to understanding how homogenous a sample is. Instead of emphasizing THC at time of consumption, it could emphasize THCA at time of testing. It could include terpenes. There are a lot of ways the state could disrupt the fixation on THC by adjusting cannabis labeling.

The Labs Could Create an Organization to Self-Regulate

I actually thought that this was pretty likely, but now I’m skeptical. It’s a graceful-seeming solution: a laboratory coalition could pool resources to run regular lab round-robin events, make self-governing rules regarding standards that go above and beyond the state minimums, and help to educate consumers and market the coalition itself. Much like the American Medical Association sets standards for doctors, such an organization could establish standard advise the OLCC on market trends in addition to creating internal standards around sampling, calibration, and other topics that regulatory bodies such as the OLCC and ORELAP are unable to realistically create and maintain in such a quickly evolving environment.

The reason I’m not sure if it’s likely is that I don’t think that many of the laboratories feel threatened enough to require them take any definitive steps towards working with other labs that they see as direct competition. Without extraordinary amounts of pressure to overcome their distrust in each other, these efforts are likely to fail.

The Cannabis Industry and OLCC Could Increase Consumer Education

It’s easier to find misinformation or overstated benefits of cannabis than good cannabis education. The industry itself could invest in programs to increase consumer education around cannabis. The industry could help consumers understand the importance of terpenes, cannabinoids, VSCs, and other compounds to the cannabis experience. They could give educational webinars on how growing methods impact the cannabis produced, and display information on studies regarding the impact of THC to the cannabis experience.

The OLCC could also invest in consumer education. Their materials They could invest in Point of Sale materials that explain exactly what that THC number actually means. The information I gathered for this article took scouring their website and administrative rules — and making this information as easily accessible as the OLCC’s instagram page might give consumers the insight into THC that they need to make appropriate purchasing decisions.

State-established Rules Could Designate or Establish a Reference Lab — Even a Rotating One.

One of the biggest flaws identified in the 2019 audit of Oregon’s Cannabis labs was the lack of a reference lab. The state could create a reference lab, designate current laboratory to serve as the reference laboratory for cannabis laboratories, or could designate a cannabis lab to serve as the reference lab on a rotating basis.

A reference laboratory is a laboratory that serves as a check on test results, helps set standards, review procedures. It can also help train laboratory staff, create certification standards, and assist in the standardization of laboratory data across the state via reference libraries.

Consumers Can Exert Pressure

As consumers, we don’t have a lot of power in this situation, but there are some key things we can do. We can educate ourselves about what the labels tell us, and what we are seeking from our cannabis experiences so that we no longer order cannabis by the THC amount. We can ask for more information about our cannabis when purchasing, including lab that tested the product, terpene profiles, RSD, and other information to demonstrate interest in metrics other than THC. We can even go so far as to decline to purchase from brands that use particular labs or do not share enough information. While we have limited resources and recourse, we can change our buying habits.

Without meaningful regulation that includes appropriate audits and efficient controls, honest laboratories will be left unprotected in a market driven solely by escalating demand for inflated THC numbers and fraught with rogue actors. Without action, public and industry trust in cannabis laboratories will continue to erode.