FDA Validation of Companion Diagnostic (PCR test)

I’ve been helping get a companion diagnostic get approved as a FDA test. This blog series will describe the statistics (R) used for the FDA validation of the companion diagnostic.

The companion diagnostic test is a PCR test to check for a fusion / rearrangement on the customer’s DNA. Normally, people have gene A and gene B. If the patient’s DNA shows a fusion between gene A and gene B, then she should be administered a drug that is tailored to the A+B fusion.

The lab did the experiments and I performed the statistical analysis in R. This series of posts will describe the analysis.

Note: This isn’t production-level code, but it’ll do the job.

Experimental Setup

The PCR test measures FAM and Texas Red values. There are 42 PCR cycles, and so if a fluorescent signal is observed, the Ct signal must be below 42.

The FAM is the signal for the rearrangement — if it’s detected, then the Ct value (the PCR cycle that it’s detected in) should be less than 42. If the fusion is not present, then the signal is never detected and the FAM Ct value is NaN.

The Texas Red value is the signal for wild-type or normal gene A. It is the internal control — to make sure that DNA is in the test sample, we expect some normal DNA in every test because the fusion is heterozygous and a cancer sample may not be pure. We will figure out the reportable range for Texas Red in the Reportable Range Section .

The Cy5 channel measures the presence for wild-type or normal gene B. it is another internal control.

The method we used to decide whether the fusion is present is delta Ct = FAM-Texas Red.

(You could also use FAM signal only if you didn’t want an internal control of wild-type DNA, it depends on how your validation plan was written.)

FDA validation is a long process that takes a few hundred experiments to examine:
Part 2. Pre-processing
Part 3. Analytical Specificity
Part 4. Accuracy
Part 5. Run Control Specification
Part 6. Reportable Range
Part 7. Limit of Detection (LoD)
Part 8. Precision (repeatability & reproducibility)
Part 9. Checking Controls


Creating Your Own Personal Silent Retreat

I was seeking silence and solitude so I spent a few days at Bali Silent Retreat. It was an ascetic vegetarian lifestyle (which was a bit much for me). While I had my moments of revelation, I realized I didn’t have to fly to Indonesia for silence. This blog post will describe how to create your own silent retreat.

The Bali Silent Retreat is not actually silent. Teachers instruct yoga and meditation, guides talk on tours, and you are asked to chant during ceremonies.

The silent part is removing the noise in your life. For most of us, this is our electronic devices including Internet, and social media. We spend a lot of time on our email accounts, Whatsapp, Telegram, Facebook, Instagram, and Twitter, etc… Noise can also come from family/roommates if you don’t live alone.

The first thing we did at the Silent Retreat is to lock up our electronic devices. There are no electrical outlets in the rooms and there’s no wi-fi signal anywhere.

That’s it! It’s that simple. The secret sauce to a silent retreat:

1) Locking up your electronic devices
2) Avoid excessive talking and stress
3) Spending time alone with your thoughts

If you can do this for a day or two (or even half a day), you’ve created your own silent retreat.

You might be asking: If I can’t go on the Internet or talk to people, what do I do?
The silent retreat had a lot of optional yoga and meditation classes to fill up the day.

Bali Silent Retreat’s Schedule
6-6:45 am Meditation
7-8:30 am Gentle Yoga
8:30-10 am Breakfast
12-2 pm Lunch
2-3:15 pm Yin yoga
3:30-4:30 pm Meditation
4:30 pm-6 pm Dinner

But these classes were optional. You don’t have to meditate or practice yoga — you can do whatever activities that are enjoying and calming to you. Some activities could be:

  • hiking
  • cooking
  • yoga
  • sewing
  • gardening
  • massage
  • taking a bath
  • reading inspirational books

Fill up the day with activities that are familiar, calming, and personally enjoyable, so you can let your mind meander and wander.

Calm your mind first; stillness and insights will come later.

If you feel antsy, write it down for later (using pen and paper), then let it go. This is your time, for you.

You’ll also eat alone in silence and without distraction (no devices nor reading). This helps maintain the silence.

Here’s the reality.

Silence at home
I admit silence is hard to achieve at home because my instinct is to check the Internet every 10 minutes. However, I have still been able to find a 10-hour period of time of silence. I do this when my roommate is travelling so the house is quiet. I work a half-day in the morning so I can check all my emails and get everything out of the way. Because everyone at work knows I’m taking the afternoon off, they won’t expect me to reply until the following day. When my period of silence starts, I make sure nothing is hanging over my head and shut off all devices. And then I fill the rest of the day with activities that I enjoy (see list above)

Silence when travelling
It’s much easier for me to attain ‘enlightenment’ when I travel alone. When I travel, I only use the hotel’s Internet, so I’m unplugged. Importantly, people don’t expect me to respond when I’m travelling. I’ve attain silence similar to what I had in Bali Silent Retreat in a museum in Washington, D.C., soaking in Calistoga baths, or wandering Central Park in New York.

I hope this helps you find your inner self.


Fantasy and Genetics – Do They Go Together Like Science and Fiction?

This could be a disaster. I’m launching a new website that does genetic analyses for fun. It combines make-believe and science.

There are a lot of direct-to-consumer genetics products out there, yet genetics is not very predictive at the individual level. The science behind these difficulties is abtruse, so why not have some fun while helping people understand their DNA better?

In between my various consulting jobs, I caught up on TV, movies and books. Specifically, I got hooked on Game of Thrones.

Game of Thrones has a lot of drama, but it also has genetics! It’s an important plot point – one of the heroes Ned Stark ends up dying because he understands genetics!

Ned Stark recognizes that the black-haired King Baratheon can not be the father of Queen Cersei’s golden-haired children, which means her children should not inherit the throne.

The seed is strong… All those bastards [from King Robert], all with hair as black as night. … No matter how far back Ned searched in the brittle yellowed pages, always he found the gold yielding before the coal.

Ned confronts Queen Cersei with his genetic discovery; which leads to Ned’s downfall and eventual execution.


After getting over the shock of Ned’s death, I went meta and realized that genetics and fantasy don’t have to be completely separate — they can be mixed.

Our favorite characters have interesting personalities and traits, and a lot of these characteristics have been researched. Based on this research we can create a genetic sketch of what our favorite heroes might look like at the DNA level.

My app then takes your DNA (from direct-to-consumer DNA companies like 23andMe), and predicts which traits you might have (like loyalty, brave, strength endurance). Finally, the app shows what traits you could have in common with your favorite fictional characters.

For example, the T allele in rs6265 in the BDNF gene is associated with resilience. So if your DNA has the genotype TT for rs6265 , we’d predict you were also resilient, and would likely share that trait with Ned Stark.

I admit this is one of my stranger ideas, but there is a long tradition of combining science with fiction; why not add genetics to the mix? Since I don’t have a full-time job, I figured it was a perfect time to try new things and a perfect time to fail.  I realize this may not be for everybody — but if you want to see what you have in common with your favorite fictional characters, please try out my DNA Fiction website.


Obsessions in Alzheimer’s

This video shows a woman with Alzheimer’s who is obsessed with finding and helping a non-existent cat. The obsession happens over a two-month period and you can tell she’s very distressed. She is always looking for a cat — wandering the streets and even breaking windows to go out and find this cat. Her daughter (Mulligrubs1) is very patient and reassures often that the five cats they own are safe.

OK, I’ll throw this out there —  I wonder if Mulligrubs1’s mother was using a cat as a metaphor for her mind. Why? Because it resembles what Martin Slevin, another Alzheimer’s carer, described in his own  book. His mother talked to and obsessed over an imaginary girl in the radiator. Towards the end of the book, he realizes that his mother is the little girl in the radiator, trapped, alone, and scared. Once he makes this connection and talks to his mother’s obsession goes away.


In the video, Mulligrubs1’s mother describes a cat:

  • The cat is often lost and outside. Maybe she feels her mind is lost and wandering?
  • The cat is described as wet. Cats don’t like being wet, so this means the cat is unhappy and uncomfortable.
  • Mulligrubs1′ mother always wants to lock the cat in her room. By bringing the cat/her mind in her room, maybe she would feel safer, and her mind wouldn’t wander.

In their conversations, Mulligrubs1 is constantly reassuring the mother about the 5 real cats, but Mulligrubs1’s mother constantly refers to a cat that isn’t there. Maybe some Alzheimer’s patients create a metaphor for how they feel and if we can enter their reality, we can relate to how they feel. And maybe, just maybe, knowing that someone else understands can help our loved ones?

I imagine a possible conversation between Mulligrubs1 and her mother:

Mulligrubs1: Please sit down. Tell me about the cat. I would like to help you.

Mulligrubs1’s mother: <Describes the cat as small, lost, outside, and wanting to come back in.>

Mulligrubs1: That sounds scary. The cat must be very scared. <hugs her mother> Do you feel like that cat?

I wonder what would have been her mother’s response. If it would have helped her to know that someone understood how she felt. Would she remember this or forget and return to her cat obsession? (For Martin, once he made the connection and told his mother he understood, his mother’s obsession went away.)

Of course, it’s easy to make this suggestion from my armchair. Mulligrubs1’s patience and love is admirable (watch the video to see I mean); I hope I can do the same.

Disclaimer: These are just my ponderings…


Can’t lose weight with Belviq?

With the holidays nearly over, I have to admit I’m feeling sick overeating with the holiday parties and goodies given as gifts.

We’ll be examining the gender differences in the weight loss drug Belviq (generic name lorcaserin).

Based on online ratings, Belviq works better in males than in females (men gave 1.18 more points for effectiveness than women (p=0.000064) and 1.17 more for satisfaction (p=0.000086)).

In 43% of the reviews written by men, they said Belviq (Lorcaserin) was “reducing cravings”. In contrast, only 9% of the reviews written by women said Lorcaserin “reduced cravings”. In 34% of women in their reviews said they experienced “headaches” while taking the drug, whereas only 3% of men said they experienced “headaches”. Meanwhile, 87% of men in their reviews said the Belviq assist for “weight lost” as compared to 61.3% of women. So, we can conclude from the reviews that Belviq (Lorcaserin) works better in men than in women.

Here is a comparison of the words found in reviews from men and women:

Phrase Female* Male** p-value
“reduced cravings” 9% (4/44) 43% (13/38) 0.0006
“weight lost” 61% (27/44) 87% (26/30) 0.02
“headache” 34% (15/44) 3% (1/30) 0.002

* % of female reviewers that use the phrase
** % of male reviewers that use the phrase

So, if you’re a woman not seeing great results with Belviq — you’re not the only one! Based on experiences of WebMD reviews, Belviq isn’t as effective for women.


Treating depression: Wellbutrin is more effective for females than males.

Millions suffer from depression, so it’s important to know which treatments work.

Women are underrepresented in clinical trials. Thus, it is possible that they react differently to drugs. To study this, we downloaded online drug reviews to see if women responded to drugs differently than men.

Surprisingly, women rated Wellbutrin XL higher than males. Wellbutrin is typically prescribed for depression. Women gave a +1.16 higher score for effectiveness (p=0.004) and +0.83 higher for satisfaction (p=0.045) compared to men.

In the reviews, women write that Wellbutrin XL is “effective”. Over half of the reviews written by women described Wellbutrin as “effective.” In contrast, only 1 out of 5 men described Wellbutrin as “effective.” Meanwhile, 44% of men in their reviews said Wellbutrin was “not effective.” So we can conclude from the reviews that Wellbutrin works better in women than in men.

Here is a detailed comparison of the word frequencies “effective” and “not effective” in men and women:

Phrase Female* Male** p-value
“effective” 54% (13/24) 19% (3/16) 0.025
“not effective” 17% (4/24) 44% (7/16) 0.06

* % of female reviewers that use the phrase
** % of male reviewers that use the phrase

Thus, we can conclude that Wellbutrin is more effective in females than in males.

— Herty Liany and Pauline Ng


Lighting a Path Forward

About 2 years ago, I accidentally wrote a children’s book. (Accidentally is a strange word: rather, I never intended to write a children’s book, but when an idea sticks with me, the only way to get it out of my head is to do it.)

My significant other is part of a company that makes circuit stickers. Sometimes after dinner, I would beta test his products. I would do this in my spare time, one hand fumbling with a sticky circuit, the other hand clicking for cat clips on the computer. Once the work was done, I’d tuck away the circuit stickers along with the knowledge of how to build these circuits.

And then one night I dreamt of a dark world, and a poem about bringing light to a world full of darkness. The poem stuck in my head. I wrote it down, and from there an idea grew…

And now, 2 years later, *drumroll* I present my STEAM electronics kit: “When Thea LED the Way”.

This is an interactive craftbook that comes with electrical components. The heroine’s name is Thea Prom, a take on Prometheus, the Greek Titan who brought fire to mankind. Likewise, the reader helps Thea bring light to her darkened world.

In the craftbook “When Thea LED the Way”, the reader enters Thea’s world and becomes part of her journey. The reader must help Thea bring light to her world by building circuits. As Thea grows and gains confidence in her knowledge of circuits, the reader does as well.

Of course, the kit comes with everything you need to help Thea with her journey (circuit stickers, copper tape, etc). It’s like the movie “The NeverEnding Story”, where a boy reading a fantasy book has to actively participate in in order to save the fantasy world.

I’m thrilled to see the book in print and the kit being manufactured. In the process of getting to this point, I had to oversee the design many different parts, from the electronic components, to the package design, to the book itself.

When I wrote the SIFT algorithm, my short-sighted goal was to get a publication so I could receive my PhD. Thankfully, my PhD advisors convinced me to make a website so that the research and medical community could actually use SIFT. At the time, I didn’t see the importance of deploying the algorithm. Only later when I saw the utility the SIFT website had in other people’s research, did I realize the impact of deploying an idea.

So while I am excited that a factory is ready to manufacture this kit, I realize that without a website to get the word out, the project would have little impact. Thus, I’m doing a crowdfunding campaign for this project. If you can help me spark confidence in children who might normally not consider engineering, then this craftbook would have achieved the impact I’m hoping for.




There’s No Glass Ceiling if you Leave the Room

My friends have been asking me why I resigned from my current job, with no job lined up.

It started when ants found their way into our home. I could see a solid trail of ants determinedly racing towards a destination (turns out someone had spilled soda underneath the fridge).

The ants looked something like this:

Marching in a single file, the ants were focused and targeted on their sugar goldpile at the end of their trail. There were many of them, and they were undeterred. Like in a well-established field of research, everyone raced toward a final goal because they knew the reward was guaranteed.

After we cleaned up the soda, we saw a huge dropoff of ants in our home. However, one explorer ant (“scout ant” if you’re an entomologist) found a fragrant apple, and established a sparse trail leading a few other pioneering ants to our apple. This is akin to finding a promising, new research area. You’re not guaranteed success, but if you do, then many ants will eventually follow.

I have been lucky to be an explorer ant throughout most of my career. I established burgeoning fields and developed disruptive technologies.

The Job I Left Behind

When I started my job at GIS over 7 years ago, my job was funded by so-called “hard money” — guaranteed funding — which allowed me to be an explorer ant. I was given resources and allowed to try new ideas, so I executed some pretty interesting projects. Besides continuing research in genetic variation [1,2,3], I studied whether one could use online drug reviews to measure drug performances [4], created a framework for emancipating facts from papers locked behind paywalls (think SciHub with a legal theory) [5], and even looked at the patterns of Facebook memes [6].

However, the funding model at the institute changed; it started to transition to grant funding. The grant funding scheme was not guaranteed for a PI; funding instead was allocated top-down and focused on directed, big-money topics such as heart disease and cancer.

When I started writing grants, I found myself using buzzwords and adding unnecessary tasks to pad grants. Rather than designing the project the way it should be, I would bloat projects to consume the available funding.

The environment was changing, too. When resources are tight, people’s behaviors change. I looked around and saw certain behaviors emerging as people struggled to survive – this is not what I wanted to become.

Inevitably, I started to change: my science was changing and my efficiency was changing. And I didn’t like this change.

Large projects were winning funding, but I couldn’t be passionate about them because they were not risky or groundbreaking enough for me. Funding agencies were after success metrics like number of patents filed and licensees signed. They wanted a sure win, just like the ants rushing to our sticky soda sugar mine.

However, going into a heavily researched, well-established area? It’s just not me.

I could no longer be an explorer ant, and was worried I’d become:


I knew what I didn’t want to become, but what did I want to become?

I went through a lot of soul-searching. I applied for jobs, but nothing seemed to click.

A couple of incidents happened around this time.

  1. I heard a female entrepreneur speak, which is rare sight. She was realistic, open, and honest about the challenges in her life. What impressed me was that she had defined her own unique path.

    Instagram: lsjourney

  2. On a long flight, I watched the movie Moana on my dinky little airplane TV. It is about a girl finding her own identity (and saving her village in the process). These are my favorite lyrics in the whole movie:Like Moana, I have “journeyed farther” and have accomplished a lot in my career. My environment may change, but I should not let it change me. I still want to explore, to push boundaries, and to do ground-breaking things.
  3. My partner heard me singing “Moana” so often that he knew the lyrics without seeing the movie. Out of love, or perhaps to salvage his own ears, he suggested I take some time off. The way he put it was “if I didn’t make you happy, you’d leave me in a heartbeat. So if your job doesn’t make you happy, why not take a break instead of holding out for that new right job to come along?”

So I resigned.

The Future

(This is the shortest section because my story is still being written…)

So yes, I am unemployed! I am taking a step back to look at the big picture. I ain’t gonna lie: it’s scary like jumping off a cliff without a safety net.

Instagram adrenaline.addiction

Since my resignation, I’ve been rediscovering me. I have worn many hats: industry, academic, and clinical. I have always been defined by a role; now I will define my own role.

I have a general strategy. I’ve accepted a few part-time consulting jobs that I’m excited about, because it lets me work on exciting projects in new areas with amazing people. I am also very lucky to be a bioinformatician because all I need to continue creating is a laptop and access to cloud-based supercomputers.


Being a consultant is like being a tour guide –bringing others to newly discovered areas. I’ll also use some time to explore the unknown.

Most importantly, I’m keeping time for myself. There are some personal things on my bucket list I want to tick off. Some are genomic and disease-related; some are not. But in the end, I want to find something that challenges me and grows me.

Sure, like any explorer, I may fail.

But I’ll never know if I don’t try, right?


1. SIFT missense predictions for genomes. Nature Protocols 11:1-9. []
2. Phen-Gen: Combining Phenotype and Genotype to Predict Causal Variants in Rare Disorders. Nature Methods 11:935 []
3. Predicting the effects of frameshifting indels. Genome Biology 13:R9
4. Assessment of Web-Based Consumer Reviews as a Resource for Drug Performance S. Adusumalli, H. Lee, Q. Hoi, S.L. Koo, I.B. Tan, P.C. Ng (2015) Journal of Medical Internet Research 17:e211
5. Enabling Public Access to Non-Open Biomedical Literature via Idea-Expression Dichotomy and Fact Extraction. Association for the Advancement of Artificial Intelligence Workshop on Scholarly Big Data [FactPub]
6. Information Evolution in Social Networks. The 9th ACM International Conference on Web Search and Data Mining


Reverse engineering contingency (2×2) table from Odds Ratio (OR)

Given the odds ratio (OR), we will calculate the individual cells in the contingency table (a,b,c,d).

In yellow, I’ve highlighted what is known.
a,b,c, and d are unknown and what we want to calculate.

Odds Ratio = (a/c) / (b/d)

Cases Controls Total
Exposed a b total_exposed
Unexposed c d total_unexposed
Total total_cases

If you’re getting the OR from a paper, the paper usually has total_exposed, total_unexposed, total_cases,and total_participants.

In that case, you can derive a, b, c, and d.

Solving for a:

Cases Controls Total
Exposed a total_exposed – a total_exposed (a+b)
Unexposed total_cases – a total_unexposed – total_cases + a total_unexposed (c+d)
Total total_cases (a+c) total_controls (b+d) total_participants

So now, the equation for OR can be written in terms of a and the known numbers :

OR = (a * d) / (b * c)
OR = (a * (total_unexposed – total_cases + a)) / ((total_exposed – a) * (total_cases – a))

If you have the values for OR, total_exposed, total_unexposed, total_cases, and total_controls, you can solve for a</i> using the quadratic formula.

Once you solve for a, solving for b, c, and d is trivial.

Try it out!

 Deriving cells of 2×2 Contingency Table from Odds Ratio:

Enter values in yellow cells


Absent Present Totals
  Group 1  
  Group 2  


I came across this problem when reading an Alzheimer’s paper.

Looking at ApoE ε4 carriers (n=452), smokers have an OR of 1.97 for dementia compared to non-smokers.

Because this was a population study, I wanted to know how many smokers got dementia, and how many non-smokers got dementia. If I got the individual cells, I could calculate this.

Out of the 452 ApoE ε4 carriers, 207 were smokers (45.8% of 452) and 31 had dementia (6.9% of 452).

From this,

  • OR = 1.97
  • total_exposed = 207
  • total_unexposed = 245
  • total cases (those with dementia) = 31
  • total controls (without dementia) = 421

I plugged in the above calculator to get:

Dementia Non-Dementia Total
Smoking 19 188 207
Non-smoking 12 233 245
Total 31 421 452

In this population-based study, 9% (19/207) of the smokers had dementia while 5% (12/245) of the nonsmokers had dementia.


Alzheimer’s book: 100 Simple Things You Can Do to Prevent Alzheimer’s and Age-Related Memory Loss

The book “100 Simple Things You Can Do to Prevent Alzheimer’s and Age-Related Memory Loss” was written in 2010.

You can find all of the advice on the Internet, and in more succinct terms.

The author Jean Carper lists “100 actions” you can take to prevent Alzheimer’s. Each action is written as a chapter. My biggest pet peeve was the repetition of the list. It’s like she thought the reader had memory loss and so repeats the same thing for 3 or 4 times.

For example, one recommendation is to ‘use your brain’. OK, got it. She repeats this theme at least 3 times as part of her list of 100; chapters “Build Cognitive Reserve”, “Get a Higher Education”, “Google Something” are all the same.

Another example: exercise to improve memory. This message to exercise is repeated in chapters “Be a Busy Body”, “Prevent and Control Diabetes”, “Enjoy Exercise”, “Avoid inactivity”, and “Watch your Waist.”

The irony is that this book is targeted for an ‘intelligent’ reader — it mentions a lot of scientific studies. I appreciate that, but if your reader can understand words like insulin and inflammation, then don’t you think the reader would notice filler pages?

There are not 100 things — maybe 20 things at best. It’s like the editor told the author — we can only sell a booklet of tips for 30 things for $5. But if you can make it a list of 100, then we can sell the book for $20. Boo! Hiss!

Jean Carper did mention some scientific studies that I’d like to follow up on, especially those specific to ApoE4.

Specific for ApoE4

  1. Alcohol is bad for Alzheimer’s

    8-14 beverages / week : 37% lower risk of dementia, but this does not apply to E4
    > 14 beverages / week : Doubles the odds for Alzheimer’s
    Adults who usually drink lightly or moderately, but go on occasional binges are 3x more likely to develop dementia
    Drinking with ApoE4 pushes Alzheimer’s 4-6 years earlier

  2. High homocysteine levels and ApoE4 is bad. Taking vitamin B can lower homocysteine levels
  3. Mice genetically destined to get Alzheimer’s (check if ApoE4) fed with nicotinamide (over-the-counter form of niacin) zipped through mazes and did not get Alzheimer’s. There are studies in progress to see if niacin-packed foods delay Alzheimer’s. Eat niacin-rich foods like tuna, salmon, turkey breast, sardines, peanuts, halibut, and chicken
  4. Don’t eat fast food (tested this in ApoE4 mice)
  5. ApoE4 carriers are especially susceptible to tiny blows to head
  6. Wear a nicotine patch for ApoE4 – cognitive boost greater in carriers of 2 copies hmm, don’t know how I feel about this

Treatments/Foods for Alzheimer’s in general

Ways to Measure

  1. Ankle-brachial index (ABI) test: leow ABI readings likely to get vascular dementia / Alzheimer’s
  2. Measure C-reactive protein. Keep it low (if you have too much inflammation, C-reactive protein levels become high
  3. Balance (how long can you stand on one foot)
  4. Systolic blood pressure > 140 mm in midlife, stronger predictor of demenita. Ideally,
    < 120 systolic, > 80 diastolic
  5. Keep homocysteine levels low or take vitamin B
  6. PET scans show deposits before any signs of mental impairment

Older posts «