TAZ has been running Threadloom search for some time now, and I thought it would be nice to hear more about it from Paul Lee - CEO of Threadloom.
Thanks to Paul for taking the time to share information with us!
Tell us a little bit about yourself.
My dad was a minister and my mom was a computer programmer, so, unsurprisingly, my career has been a mix of social impact and technology. After college, I consulted on enterprise software, then spent three years working with nonprofits before going to graduate school and joining Google as a product manager.
My family jokes that I have a PhD in shopping , so I was pretty happy with my first gig at Google in Product Search. I learned about search, started Local Shopping, rotated through Google.org (with three other Threadloom team members!), and spent my last stretch working on improving Internet access in emerging markets. My last two projects were bringing Internet access to Indian train stations, and eliminating buffering from YouTube.
I left Google in the spring of 2015, and have been working on Threadloom since then.
What is your educational background?
I was a public school kid up until college. I went to Stanford for my undergrad and masters, and Harvard for business school. At Stanford, I studied international relations and policy studies. Lots of fun, but the jury's still out on how useful it was career-wise. Maybe it will pay off in 20 years when Threadloom has customers in tons of countries.
On a side note, I'd known throughout my childhood that there had been some complications when I was born, but I didn't know the details. When I got my college decision letters, my parents filled in the blanks. Apparently, the doctors told my parents that I might have brain damage. When I got into college, my mom said, "Now I know you are okay!" True story.
What is your fulltime job?
I'm the CEO and co-founder of Threadloom, and am blessed to work here full-time. In a company as small as ours, this means that at any given point during the day, I might be talking with customers, working on product design, or assembling furniture. I love it (and those things in that order) and have never been happier.
How did you get involved with Threadloom?
I grew up using BBSes and forums. While at Google, I noticed that forums had a staying power that was really impressive. After almost eight years at Google, I left with an interest in learning more about how forums work, what their challenges were, and how I could become involved. Many owners and contributors were gracious with their time, and shared about their motivations, frustrations, and lessons learned.
One thing that kept coming up over and over was how broken search was, and how hard it was for people to find content. I knew from my time at Google how important search was for getting users to come back to your product, and for driving ad revenue. In forums, people were spending more time searching than finding.
So I ran a little experiment. I asked some of the forum owners I'd spoken to if they'd let me analyze their search queries, and they agreed. When I saw the results, I was floored. I wrote about this a little on our blog, but the short summary is that a ton of searches were yielding no results. And it wasn't because the content didn't exist. No one had built a search engine tuned for forums, and as a result, users were missing the best content in the forum.
At around the same time, new tools and platforms were coming online that made it possible to build a very powerful cloud-based search experience. Having used BabyBargains.com when we were expecting our first child, I spoke with Alan Fields, explained what we wanted to do, and asked him if we could build a new search engine for him. He agreed, and in January 2016, BabyBargains.com became our first customer.
What is the basic architecture of Threadloom?
We offer PHP plug-ins for vBulletin and XenForo. These connect to our cloud search engine that sits on top of Google Cloud and Microsoft Azure. Our engineering team is mostly from Google and Microsoft, so they have previous experience working with these types of large-scale cloud services and data pipelines.
But we also wanted to make Threadloom super easy for any forum owner to install and use, so we set a goal of building a completely command line-free experience. Due to the diversity of hosting environments and setups, we're at about 95% of that goal. For these customers, installing Threadloom is just a few clicks in AdminCP.
We also knew that with cloud search, it was possible to occupy a very efficient resource footprint on the client server, so we designed Threadloom to use very little memory and CPU, and we're happy that customers are noticing the lightened load on their servers.
What resources does it use on the backend?
We use containerized deployment environments to build and test across various combinations and versions of vBulletin, XenForo, PHP, MySQL, and web servers. Our data pipelines use cloud storage, auto-scaling, and messaging (like Google Pub/Sub) to ensure we process all incoming data in a lossless fashion.
Our search stack is an advanced version of elasticsearch that we've tuned for forums. It uses Azure's search stemmer and a custom scorer, as well as additional modules for spell correction, machine learning, and natural language processing (for our More Like This experiment).
What benefits do users gain from Threadloom?
For forum members and guests, the biggest benefit is that it's easier to find the best content in a forum. Everything we design in Threadloom centers around this, e.g., our advanced filters, the ability to preview top results from a thread with one click, and spell correction. We've found that if users have a great search experience, they're more likely to come back to the forum.
We think that one key is balancing high-quality relevance with comprehensiveness and freshness. Users don't just want a few great results, they want all of the great results. Unfortunately, native search does okay on coverage and freshness, but poorly on relevance; and Google Custom Search does well on search quality, but poorly on coverage and freshness. With Threadloom, you get great search quality, total coverage, and instant freshness.
For forum owners, there are even more benefits. I've already mentioned ease of installation and saving money on server resources. Threadloom also automatically adapts to each individual forum, respecting forum permissions and adapting the UI to style variables. Finally, there's no maintenance, and if there is ever a need, it can be uninstalled with one click.
The net result of all of this is growth: more searches, longer sessions, repeat visits, and new members. We've had very positive results from our early customers, and we have a number of improvements we're working on to drive even more growth.
What's in it for Threadloom?
We have both philosophical and business motivations.
The web is amazing, but in recent years, it's become clear that something is deeply wrong. Its primary gateways – search and social media – care more about popularity and money than quality. Forums are the one place where (at least most of the time) you can find a credible authoritative opinion . By making the best forum content easier to find, we want to help shift online discourse from the wisdom of crowds to the wisdom of expert communities.
We also think forums make good business sense.
First, it's a large market. There are more than 100,000 active enthusiast forums, and contrary to popular opinion, forums aren't dying. The sub-sector is stable, many key metrics are very strong, and forums enjoy valuable first-mover advantages like SEO history.
Second, we're working with some of our early customers on exciting new products and features. Some of these include direct monetization (like search ads), some are indirect. It's still early, but if these experiments succeed, there will be plenty of opportunities to collaboratively create value for many users.
What information does Threadloom collect from the sites, and how is it used? Is it ever distributed to third parties?
We construct each forum's search index using the forum, thread, post, and user tables. We sync this data using a secure, encrypted connection to protect each customer's data.
For the end user, we log searches and use them to improve the product. We don't collect session IP addresses or personally identifiable information. We also don't distribute or sell any information to third parties.
Are users ever redirected to other (potentially competing) sites?
No – and we've built in a safeguard to keep us accountable from actions that aren't good for forum owners. We designed Threadloom to be easy to install, but we also designed it to be even easier to disable or uninstall. This means that at any point, any forum owner can easily turn us off or completely uninstall us. So it's in our interest to keep our customers happy.
We also try to share where we are going and what experiments we are trying out. During our Early Access period, for instance, we shared early mocks and experiment results with our beta customers, and invited their feedback. We plan to continue doing this on our blog, and love getting feedback from customers.
Tell us about the Threadloom team.
I love our team – it's my favorite thing about working at Threadloom. We have five full-time team members, 3 in the San Francisco Bay Area, and 2 in Seattle.
Building a meaningful culture is important to us, so one of the first things we did as a team is come up with a set of company values. Four of us (Greg, Alyssa, Evan, and I) had worked together at Google.org, so we had a head start in thinking about culture and values. But it was also pretty easy because everyone has made personal and professional choices to focus on trying to help people, and has chosen to take a bold path in order to follow their passions. I think there's a lot in common there with most forum owners I've met.
- Greg spent almost a decade at Google, leading the engineering teams for Play Analytics, Voice, Video Ads, and Google.org. Before Google, he led the team that launched the TiVo HD DVR. He loves comic books, hockey, and whiskey (not necessarily in that order).
- Alyssa was at Google for five years and worked on Chrome and Google.org. While at Google.org, she built nonprofit knowledge panels (do a search for [ijm] or [red cross] on google.com), as as well as Google for Nonprofits, which donates $1B of Google products and services each year. Alyssa's a runner and has two awesome dogs.
- Jenna comes from the startup world. She was at Tuition.io before joining Threadloom. She loves Hamilton and has had the soundtrack on repeat for the past six months.
- Evan is Alyssa's husband, and our newest team member. Evan worked on Enterprise, Google.org, and Fiber at Google before leaving last month to join us and lead product operations. Evan loves literature and old school, leather-bound books.
What's something cool Threadloom can do that other forum search engines can't?
We're the only forum search engine that includes spell correction. And because we're cloud-based, we can do some pretty cool stuff. We recognize slang and popular names, and can also correct homophones in context, like Google does. So if you search for something like "best shoes four babies," we'll suggest "best shoes for babies."
What's coming in Threadloom's future?
Leading up to our launch in March, we focused on making sure our platform scaled well and would perform well with thousands of forums using it simultaneously. Now that we're in general availability, we're making the core product even better for forum users. We recently added support for XenForo thread prefixes, and are working on image search, new filters, and search over WordPress blogs and other non-forum content.
We've also run experiments on search ads, SEO, and artificial intelligence-based search suggestions (we call them More Like This). I'm especially excited about More Like This: In our initial experiment, eCPMs went up more than five-fold when users clicked on it.
What other projects are you involved with?
My primary project outside of family, church, and work is the International Justice Mission (IJM), an anti-slavery organization that is one of my favorite nonprofits. There are more than 30 million slaves globally, half in India, and IJM is one of the most effective organizations fighting to free slaves and reform justice systems. I've had the privilege of volunteering for the past six years, and serving on the board for about half that time.
Which online sites do you like to visit?
Right now, YouTube TV. I've also been spending more time lately on Piloteers, and my regulars include Tesla Motors Club, Slickdeals, The Verge, TechCrunch, Rotten Tomatoes, the New York Times, and the Wall Street Journal.
Do you run any forums?
I run theMouthPiece.com, a British brass bands forum. The previous owner and our moderators deserve the credit for where it is today. It's been a lot of fun and a great learning experience. Some of our team members are also starting new forums, and they'll be on TAZ looking for advice and tips.
What do you like to do for fun and adventure when you're not online?
Besides shopping, I enjoy travel (though not the separation from family and friends), and our family enjoys hiking and cycling. But the truth is, I'm online too often and this question reminds me I should get offline.
What are your favorite things?
Electric vehicles. Paperless offices. Marriott hotels. Family dinner. Movies with happy endings. A capella music. Campfires. Street Fighter II. Vulnerable leaders (because I played too much SFII as a kid?) AirPods. The perfect photo. Uniqlo pants. Making my kids smile.
What does the future hold for Paul Lee?
In my mid-30s, I feared that I'd learned most of what I would learn in my career. That concern has been completely obliterated over the past two years. So I've stopped planning too far ahead and have learned to enjoy the moment more. However, I'm pretty risk-averse, so one exercise I practice is take a new risk every day, and so far, that's been working out pretty well.