Meet The Team #7: Prashant Vithani (Not the new guy, but the Μμ guy)

Welcome to another edition of Meet the Team at Clarisights. Today we're sitting down with Prashant Vithani, one of our longest-serving, engineers who has played a crucial role in building our data ingestion frameworks and core platform capabilities.

Let's start at the beginning. How did you first come to Clarisights?

Prashant: That actually connects back to my introduction to programming itself. When I was in college, I wasn't even planning to go into software. I wanted to focus on electronics! But then in my third semester, my friend Shubham Patel introduced me to Ruby during a casual conversation in class. He was late to class that day and happened to sit next to me because that was the only place he could find. That random conversation led me to explore Ruby during my vacation, and that's how I fell in love with web development.

Years later, I had an account on AngelList - I think it's called Wellfound now - and I got an email from Ankur. They were looking for someone with Ruby on Rails experience, and the assignment was to build an application in Rails. During the interview, Ankur started talking about the marketing domain, and I had absolutely no clue about it. I just kept saying "yes, yes." But somehow I finished that conversation and survived! Looking back and connecting the dots, it's amazing how that chance introduction to Ruby in college eventually led me to Clarisights.

That sounds a bit less technical than our interviews today. What was it like when you first started?

Prashant: I came in as an intern. There were five other interns who started at the same time. The first week was really about just trying to be comfortable. I was a bit afraid because I didn't know anything about this domain. Everybody here kept talking about digital marketing so confidently, and I didn't have a clue about what happens after somebody clicks on an advertisement.

But then Raghu from our Customer Success team sat with me. He explained everything from scratch: how ads are created, what marketers do, why they create ads, what the objectives are, and what data is generated after the ad is created. After that, it was simple. Because I understood the problem and the domain, the technical solution became trivial.

That's quite a journey from those early days. What do you work on now?

Prashant: Today, I lead the Μμ framework team. The scale of what we handle is pretty incredible. We're running about 10,000 jobs across 1,000 channels, upserting nearly half a billion rows a day. When I talk about a single channel, the biggest file that we've processed is 17 million rows - and these aren't just new rows being appended, we're actually updating the existing data on all 17 million rows every day, followed by annotating them. And that's just one channel I'm talking about.

If I talk about the overall scale, not just Μμ, the sizes usually vary by channel and customer. But generally, the size of a single channel is in the order of 50 million rows. So when you multiply that across all our channels and customers, we're literally processing billions of rows of data every day. Sometimes I look back at my first task at Clarisights, working on a single Facebook integration, and it's amazing to see how far we've come.

The Μμ framework is central to what we do at Clarisights. How did it come about?

Prashant: The Μμ framework came from learning from our earlier mistakes - mistakes that I made, actually! After my internship was done and we decided to expand beyond Facebook, we started integrating with AdWords, Twitter, Bing, and others. At that time, I thought I had this brilliant idea. Rails provides these generators where you run a command, and it generates a bunch of files with a template. So I wrote generators that would create about 30 odd files for each new integration.

At the time, it seemed like a great idea because it reduced the time to build a new integration. But after two years, when we had about 10 integrations, I realised the mistake I'd made. What happens when you keep using generators is that you end up having a new product for every new integration, all built separately, and there's no sense of framework for data ingestion or ingestion pipelines.

So in 2022, we launched a new project to build a general-purpose ingestion framework that follows a set of instructions - basically configs - and processes and transforms the data as per the configuration. In a way, my job today involves rectifying mistakes I made some years ago. But that's the beauty of engineering at Clarisights - we're constantly learning and improving, and we're not afraid to admit when something needs to be completely rebuilt.

Recently, the engineering team at Clarisights has grown significantly. What makes our engineering culture special?

Prashant: Every Monday, I look forward to coming to office. I've noticed people from other companies are always looking forward to holidays or counting down to the weekend. Here, we have this working relationship with colleagues where it genuinely doesn't feel like work.

Tell us about your colleagues. (You are famously close to fellow engineer Neel Maitra)

The co-workers here are amazing - in fact, we've become friends and we hang out outside of work too. But it's not just about friendship. You need to have intellectual conversations, too. So, these are very rewarding relationships.

There is a core group - people like Anmol, Neel, and Pritam - who've stayed with the company through various phases. Anmol and I used to go to Cult classes together. Later, we also went on a workation to Europe together. My friendship with Neel and Pritam developed during the POT (Per Object Tagging) project. This project is also close to my heart because it's where I learned what engineering is all about.

When Pritam joined, and he was working remotely from Mumbai, we started discussing the project remotely over the phone. He would ask me about every small detail, and that's when I realised that no detail is trivial or small. When Pritam came to Bangalore, we would sit day and night going through the conceptual design, then the application design, and POC. We eventually started working on it during the COVID era, which was again remote.

The first few months were a struggle to be honest, but eventually we got together. That core POT team transformed into a tight-knit friends circle - Anmol, Neel, Pritam and I were part of it.

Neel and I were the first two people to start the implementation of the POT system. After POT was delivered, we had this period of continuous on-call duty for ETL. POT is part of ETL, but now our responsibility expanded into the whole ETL component of the product. At the time we used to run into issues because, as I mentioned before, we had as many products as the number of integrations. Two people can't maintain all these products because they're all built differently - whoever works on integration has their own ways of writing the code.

It was really frustrating, I'm not going to lie. But our relationship was built on supporting each other through these growth pains. If I'm on call, I would solve the issues with him on the call. If he's on call, I would get on the call with him. Over time, we developed this understanding of how to pair. It got to the level where Neel and I would write Zulip messages in pairs - he would write the message, share the screen, I would review that, edit and then send. I can't imagine Μμ's development without Neel being there.

For engineers considering joining Clarisights, what advice would you give them?

Prashant: I think the key trait is curiosity - and I mean real, deep curiosity. This basically becomes the core trait of the people who work here, or who would like to work here. When something comes up, you get curious enough to not let it go. You become obsessed with it. You want to know what the root cause of an issue is. And some issues are so nuanced that finding a root cause is tricky. In short, you have to be obsessed with investigating and finding the answers.

Finally, working with our founders - Arun, Ashu, and Ankur - what's that like?

Prashant: Each of our founders brings something unique to the table. Our CEO Arun, is a really chill guy. Whenever he's in Bangalore, he goes to people's desks and just strikes up a conversation, talks to them casually, nothing formal.

If I talk about Ankur, well, I was afraid of him in the beginning! He was like a father figure who was there for discipline and strictness. But he has changed a lot. He's the person I would recommend and even go to for validation when designing a solution. And if I talk about Ashu, he's a saint. I've never seen a person who can be so calm in the most challenging situations. There's a lot to learn from him.

Between Arun's deep understanding of performance marketing as a business, Ashu's theoretical and practical computer science expertise, and Ankur bridging the gap between business and technology, it's a really powerful combination.

Any hobbies?

Thanks, Prashant. This has been fascinating. Any final thoughts?

Prashant: Just that I'm excited about what's ahead. We're continuously adding more customers to our portfolio, and that means more interesting problems to solve. And that's the reason I've stayed at Clarisights - the challenges keep evolving, and we keep finding new ways to solve them.

This conversation has been edited for clarity and length. For more information about Clarisights, visit clarisights.com. If you're interested in joining our international team, check out our latest opportunities at careers.clarisights.com.