about us

Did it Really Work? Trust but Verify

Our Mission

Sound decisions about drugs and treatment can only be made based on scientifically directed research and timely, relevant communication.

Instead, we are routinely bombarded by advertisements from drug companies. The funny thing about drug companies is that they claim that their products really work but they don't ask their customers about it. Of the billions of dollars a year that go into drug research, almost none go into researching drugs that are already marketed and claim to work. Actual patient experience has been too long ignored.

Independent and balanced evaluation of these products as they are used is urgently needed. As current or future users, we have the right to know whether the treatment lives up to expectations.


As citizens of the world, we are constantly bombarded by news of amazing cures and claims of medical wonders. As physicians and scientists, we see every single day the unfortunate gap between wishful thinking and reality.

A friend of the family, Pat, recently experienced several transient ischemic attacks, unable to speak for several hours: There was insufficient blood flow to the speech producing centers of her brain. For many years, she knew she was at high risk for stroke and heart attack, because of high cholesterol. She chose not to take cholesterol lowering medications because she wanted to avoid muscle side effects (myopathy) that can be caused by the statins.

The key information that Pat needs is the percentage of patients who experienced myopathy after taking statins. It turns out that in randomized clinical trials used by drug companies to gain drug approval, 1.5-5% of patients experienced myopathy. However, in observational studies of real world patient experiences, 10.5-25% of patients suffered myopathy. [reference: Fernandez et al, 2011]

The much higher risk of myopathy in real world studies of statin users highlights the crucial role of observational studies to verify the results of randomized clinical trials. When real patients are studied as the drugs or devices are actually used, then we will be able to verify the efficacy and safety claims made by the inventors and sellers of these products.

Another 2014 lesson came from the ignition switch defect that led to deaths and recall of more than 2.6M cars by General Motors. In December 2005, GM sent a dealer bulletin stating that the car can turn off unexpectedly when "the driver is short and has a large and/or heavy key chain." [reference: NPR March 31, 2014]

In the real world, some drivers are short and use heavy key chains. In the real world, more patients are vulnerable to myopathy caused by statins. As a society, we can no longer afford to ignore the gap between wishful thinking and actual experiences.

Methodology: Wisdom of the Crowd

No expert can solve this problem without our help. In the absence of actual data from all of us, the experts can only speculate and share their educated guesses.

The only way to change the status quo is to collectively share our actual experiences, in a systematic and scientific way. When we share our experiences, we help our friends, neighbors, and ultimately ourselves. By relying on "the wisdom of the crowd", from all of us, each of us will all be able to make more informed decisions based on actual data, and rely less on speculations and guesses.

Each one of us is a real patient in the real world. When we contribute our experiences to observational studies, we crowd-source the data that when analyzed, will allow doctors and future patients - our future selves - to make more informed decisions.

Methodology: Beyond Stars and Narratives

Many users of consumer products already contribute to online ratings and reviews. This is an example of crowd-sourcing real world experiences and highlights the fact that people are altruistic and willing to share their experiences to help others. However, due to the complexities of drugs and treatments, star-ratings and narratives are a good start but leave much to be desired. Carefully designed surveys that capture key relevant information about each treatment will allow proper analysis of the data. This is our responsibility: to maximize the usefulness and benefit to society of every contribution. As scientists, we evolve, improve, and create additional surveys to gain greater understanding of treatments over time.


Did it Really Work? (DIRW?) is an independent research organization founded to bring reliable scientific information to consumers and health care professionals. DIRW? is a member of ExoMachina Inc., a technology incubator.

The DIRW? team consists of medical experts working closely with software engineers. The chief science officer is Andrew Ho, M.D., a physician scientist with more than 20 years of experience in medical research and education. [Dr. Ho's scientific publications: Google Scholar]

To maintain independence and impartiality, DIRW? designs, conducts, and directly collects data from DIRW? study participants. We do not delegate or outsource any research to third party who may or may not be unduly influenced by drug companies or have other conflict of interest.

To support our mission, the only source of external funding that is acceptable is donation or non-contingent, unrestricted research grants. No advertisements from drug company are accepted.

DIRW? maintains full control and takes 100% responsibility for all scientific publications based on the DIRW? studies.