Balancing Trust and Safety with Growth

As discussed at length in my previous article, having a dedicated Trust-and-Safety team is key to earning your customers’ trust and protecting your brand. It’s important to protect customers as they are trusting you with their sensitive data and breaching their trust can kill your company. Additionally, your trust-and-safety team can help you ensure compliance with local and international law, and protect the company from financial risk.

All too often there is tension between the Trust-and-Safety team and the Growth or Marketing team. There are a number of reasons for this. First of all, Trust and Safety often needs to block malicious user accounts, reducing the all-important monthly active users growth metric. Trust and Safety also often refunds some percentage of transactions, reducing monthly revenue and revenue per user. This team may suggest requiring two-factor authentication whenever the user logs in, but this can cause double-digit percentage drops in user retention and growth.

Growth, on the other hand, wants to drive up these metrics. In order to do that, there’s pressure to launch features which Trust and Safety could consider dangerous or risky. For example, the Growth team may want to add a new peer-to-peer payments feature that could boost user retention but expose customers (and the company) to serious financial risk.

Meeting in the middle with checkpoints

Growth and Trust and Safety don’t have to be at odds with each other. In fact, Trust and Safety can empower Growth to safely launch new initiatives with minimal fear of externalities from bad actors. The solution is to take a more nuanced view of user behavior.

Rather than simply considering a user as either “good” or “bad”, we can consider a spectrum of badness and deploy challenges – called checkpoints – to try to retain the “better” customers and minimize the damage caused by the bad ones. This is how large online communities, marketplaces, and SaaS apps grow so quickly.

There are a number of different types of checkpoints. Some checkpoints are lightweight and have minimal impact on the user experience. Others are quite heavyweight and have a noticeable impact on the user experience and conversion rate. The trick is for the Trust-and-Safety team to deploy the lightest-weight checkpoint available to match the risk of the user behavior.

For example, if a user has a history of good behavior logging in from their home IP and home device, but they are trying to withdraw money from their account, we would deploy a low-friction checkpoint as it’s likely that the user is not compromised. If they use a new IP in a country they’ve never visited, perhaps we would deploy a higher-friction (but harder to defeat) checkpoint like two-factor authentication. The downside is this can increase your churn rate and adds user frustration due to wasted time and authentication difficulties while traveling.

Perhaps the most famous checkpoint is the CAPTCHA, which exploits the idea that humans are better at reading distorted text than computers, so an attacker’s script or bot cannot defeat the CAPTCHA; In reality, computer vision technology has improved by leaps and bounds in the past few years, and there are services to cheaply solve CAPTCHAs. Another problem with CAPTCHAs is that humans find them difficult and annoying to solve; users may churn out of your experience, and CAPTCHAs make it harder for visually-impaired customers to use your product.

An alternative to a CAPTCHA is SMS verification, which is easy for the user to do, but requires an attacker to get a new phone number every time they want to create a new fake account. This can get prohibitively expensive for spammers, since they require lots of fake accounts in order to make money.

Voice verification is a checkpoint that involves calling a customer on the phone and asking questions – a powerful way to defeat fraudsters. While expensive, it can often be deployed on a tiny subset of transactions. For example, deployed military personnel often use the same mail forwarding services that fraudsters use. Rather than blocking all purchases for these servicemen and servicewomen, we can call and ask them questions that ensure they are who they say they are and fulfill the order.

Checkpoints are also important for online communities.  When a user does something potentially risky like posting an automatically-flagged inappropriate photo or comment, you can automate a popup message educating the user about the “rules of the road”. If they continue to break the rules, we can put the user into a read-only state; they can keep consuming content but can’t continue the behavior that offends other customers.

Overall, deploying checkpoints rather than outright blocking customers or transactions is a growth-positive way to implement trust-and-safety. Since different checkpoints have different false positive and false negative rates, it’s important to be thoughtful about which checkpoint to use and when to use it. By using checkpoints, your Trust and Safety team can protect your customers, brand, and bottom line, while continuing to enable rapid customer growth.