Rails Business
Brendan Buckingham and Ryan Frisch talk about developing with Ruby on Rails and how to leverage it to build a business.
Rails Business
Challenges in App Integration and Data Synchronization
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
In this episode, the hosts discuss the complexities of app development, particularly focusing on integrating third-party data feeds and maintaining data synchronization within a Rails application. They explore issues like managing incomplete or variable data, updating data based on third-party changes, and the broader challenges of integration. The conversation also delves into the prioritization of feature requests, handling customer feedback, and the balance between building in-house solutions and leveraging external tools. They reflect on their experiences with specific integrations like HubSpot and emphasize the significance of clear product vision and efficient management of development priorities.
00:40 Dealing with Incomplete Data in Event Feeds
01:16 Syncing and Updating Event Data
02:13 Integration Challenges with HubSpot
02:37 Source of Truth Dilemma
04:22 Sales and Marketing Integration Issues
06:51 Product-Led Growth and HubSpot Integration Goals
09:15 The Complexity of Building Custom Integrations
21:09 Balancing Customer Requests and Product Vision
24:37 Conclusion: Reflecting on Integration Requests
Links:
I found myself today thinking about, what's the hardest part of building an app with rails. Okay. And the background on it is we have a. Importer that imports events from third party RSS feeds or XML feeds or a variety of other things. And integrations are hard, right? Cause you don't control the data that you get. And, so on one side you have no control on the other side, you want to enforce some sort of. Best practice, right? So for events, you want to make sure there's a day to start time, a title, a location, sort of the minimum number of things. But surprisingly, most event feeds, don't require that information. So sometimes you have it, sometimes you don't. So you have to, you know, what do you do when you're missing information? And we've made the decision that we just sort of, all right, if there's missing data that we've deemed required, we queue it up and it requires some sort of manual intervention. And that's all well and good. I think that's a fine approach. But now there's a new requirement that is. If the event, gets updated in the feed, we need to update it on our end, right? Just, you know, basically keep it in sync. Yep. Well, that's fine if all the data was there to begin with, because then you can compare one to the other and just make the updates. But when there was that interim step of having to You know, manually enter data now. It's like, well, what do you actually update, right? Do you update only the data you have? Do you wipe it out and start over, which could then require another manual step by the user? And it got me thinking, is working with data hard? Is it, is the requirements hard? Is the integration hard? And then it just sort of ballooned in my head to like, all right, I wonder like what, other people think the hardest part of building an app is, you know, not necessarily just rails, but an app in general. And then it got me thinking, how has that changed for you over time? It's just sort of an interesting thought process.
Brendan:Yeah, I like that topic. I think we actually just pushed an integration with HubSpot this week. And we ran into some things too, where this is actually taking a couple of tries to get it out. But I think the first time we pushed out the code, we had talked about a lot of this stuff. One thing that we ran into was, in this case, anyways, you've got our database in our application, and then you've got the HubSpot database. And, which one is the source of truth? You have to decide that for one. And then when you decide that, well, are there certain pieces of it that even if your source of truth, let's just say it's HubSpot. Are there certain things that can be overwritten by Your database, because they're dynamic, for instance, and they're going to update or, you know, things like that. It's been a challenge, not too dissimilar from what you were talking about with just integrating with another platform. And how do you make sure that you get it all to work in a way that is, usable and not breaking things or
Ryan:yeah.
Brendan:Making it extra work.
Ryan:Yeah. Well, I found in general that everybody wants to be the source of truth, right? It doesn't really matter what side of the integration you're on. I've yet to find any API that thinks it's not the source of truth. if you have an idea of which one that is, I'd be curious to hear it. But yeah, HubSpot's an interesting one. Are you able to share more about what you're, doing there?
Brendan:It's really nothing all that interesting. What it boils down to is that for a long time, we were tracking all of our companies and our users and our clients or whatever you want to call them in our application in that database. And then over the last, I don't know, two years we've started, using HubSpot to track a lot of that. But now we've got data in two places and we want to be able to sync it. So that's the biggest thing it's for. But of course, like anything, it's unearthed a bunch of other things like, Oh, wait, if we do that, that whole source of truth thing I brought up already becomes a thing. And then it's, a lot of questions like that, but integration itself, at least at the moment is fairly simple. It's nothing real complicated
Ryan:all right, well, let me ask you this, right? From a sales perspective, right? Does a business have an account with you or do individuals have an account with you and those individuals are users of that business?
Brendan:It's, it would be the business has an account and then they have multiple users.
Ryan:From a sales perspective, and this is something we run into, we haven't quite gotten to the integration part yet, but it's definitely on our sort of wishlist, to integrate with our HubSpot account. But every tool that we've tried to do something with in the past. This is the heart of the issue that we grapple with, which is, if you want to automate sales or marketing in any way, you need to talk to a person. But everything that we do in our app and that we track like usage and billing is at the account business company, whatever you want to call it level. So what are you thinking, right? Are you thinking user information so that you can target users or are you thinking company information? And the reality is you probably want to do both, but then it gets really complicated really quickly. And then it stops you from, making, even an incremental improvement. So you got to decide, what your goal is up front. Is it sales automation, or is it just that the sales team needs info about what the business and their users are doing at any given point in their account?
Brendan:Yeah, and another place my brain's going is you have a system that's similar in some ways where you've got businesses and users, and I've got a system that's got businesses and users, but we have different Associations and we've got different, record types. And our schemas are completely different.
Ryan:Right.
Brendan:And so we've got our own schemas. And then of course, whoever you're integrating with has a schema. And then how do you make all that work together? And that's actually one of the things we ran into as well with the HubSpot stuff was just that, we can pass along as much data as we want, but then you got to find a place to put it in HubSpot, and and that, I know I went around your question about the automations and stuff. And in our case, we're not really looking to do too many automations yet, but that'll probably be on the table at some point, cause once you get the wheels churning, then I'm sure some of these thoughts are going to happen like, okay, and now that we got this set up, well, what, how do we leverage this tool?
Ryan:Yeah. Have you heard of the, notion of product led growth?
Brendan:Yeah. Oh, yeah.
Ryan:The idea is that you help people find value and then, they self serve up into higher tier plans. And there's a lot obviously a whole lot that goes into that that I'm greatly simplifying, but that is our goal is to get our app to a product led , SaaS dream, I'll say, low touch high volume, right now we're the opposite or high touch, low volume, which can be a nice business, but all the longterm value of how we've organized our product is. And activating the other side of it, so my wish for our HubSpot integration is to be able to send some, what HubSpot would call a marketing email, I want to send data to HubSpot that will allow me to send an email in a timely, personalized way to a person who is going through their own journey, within our app and within the construct of their accounts or business, And that's a very different need than what the sales team needs. Sales team needs to know billing information. They need to know, product usage information. We have a freemium tier, so like you might want to know, what was the, what have they been focusing on? So like when you pick up the phone and call them or you're not going in completely blind, we tried intercom for a long time. That didn't work so well for us. We've tried a handful of analytics tools over the years, and they all start from a user that I know intercom recently introduced, the account notion, but that was fairly new about 12 months ago, after we gave up but I've never, I haven't yet found something that made it simple. And frankly, if we talked to Jason sweat, in the last episode or I don't know the order of that recently, we were talking about business ideas, like that's one I constantly think about, but there's also like a hundred players out there that do this sort of thing, right? I think I'd be able to find something that fit our need better and maybe I'm just. Don't know what I don't know, and that's could be interesting to, maybe our listeners, can share some of the tools that they use, yeah, I don't know. It's just, I think I find it integrations hard.
Brendan:Yeah, exactly. And I guess where my head's also going now, based on. What you just said there is the developer in both of us is like, well, I could probably just build it, right? To your point, and actually I just talked to Jason again recently, and we were talking about that similar thing that we talked about in that episode, which was, there are so many ideas, but you can't do them all. So how do you focus on the ones that are important? Obviously for you, that's locable. And it would only make sense for you to. Build this integration if it's something you could sell or something you could leverage to make lookable grow
Ryan:Yeah, I mean There's a whole slew of reasons why I might decide to build it right one is You know locable needs it and I can save money right hub spots expensive Especially their marketing tier build it in and you have more control over what it looks like. It can be customized. Like those are all interesting ideas. I've always wanted to do something in the open source community. And I feel like having a gem that did that, that, I then dog food and use for local, what could be an interesting way to get involved in the community, or I could think of it as. Hey, this is something locable could use and I'm going to build it for locable, but it's a separate product. I'm going to market it separately and it's a micro sass or a completely different, product unassociated with locable , and all those things appeal to me for very different reasons But it all comes back to like where do I want to spend my time and
Brendan:right? So i'll try to write.
Ryan:Yeah right now I just want to find a solution that I can plug in and it doesn't break my brain to integrate. And so far, a lot of these, solutions and their APIs do break my brain, even the good ones, right? I think Stripe is one of the best and it's still kind of confusing. So I think back to the original question what's the hardest part about building an app? I think integration is right at the top of my list. it doesn't always have to be hard, but I feel like the real problem, like the things that provide the most value typically are more complex and, are harder. There's obviously simple integrations that, going to break my brain, but the ones that I care about tend to, I've gone through iterations trying to solve the problem, in different ways. And I'm still not. Landing on a solution that I love.
Brendan:Yeah. And that also makes me think about another thing with integrations. I guess it's not even just integrations, but it certainly can. And usually in my experience happens with them, especially if it's an easy one, is that you, you get it integrated and it is working great until it isn't, but then you haven't touched that code in so long that it's like, Oh no, now I have to relearn it. And I guess a question I was thinking of for you specifically right now with the fact that if it's in a harder integration and maybe something you have to keep touching from now and then is that better in some ways because you're at least more familiar with it? Or is it, because I guess you're getting at though, is like you don't really want it to be hard and so it's like I'm trying to see where the, no, I
Ryan:don't wanna have to think about it. It's not right. You just want the
Brendan:work. Yeah.
Ryan:This integration doesn't. Directly lead to dollars on the profit and loss statement, and it's usually the opposite, there's a cost to maintain it. There's a cost to build it in the first place. Now, obviously the goal is efficiency, and if you can get efficiency and scale then, it's worth every penny. But if you can't, then it's just, it's a waste of time. To answer your question, for most of these things, I just want something that works for whatever I need it for. And I don't want to think about it. And I've been exploring, I can't remember the name of the company. There's a company that does just integrations, . And they maintain them for you. And we're thinking about just hiring them, let them manage it. I don't want to think if I have to work on a Google business integration again. Right. Like I just, it's not a friendly interface. They don't have any way to test it. Same with Facebook. Facebook's a little bit better. Google, like to test the Facebook app, their terms of use pretty much say you can't create multiple accounts for Facebook, but their support openly says, Oh, well just, we don't have a test environment. So just create a dummy account. I'm like, okay. To which I've have closed by
Brendan:them, I've done the same thing. Right. And they get closed
Ryan:Right. So it, it's just, it's, and their approval process is just painful. No feedback. Mm-hmm. Or documentation, you're just sort of guessing half the time. I'm sure there's experts out there that are dialed into it and understand how it works. But to your point, if you don't see it all the time, like I had to, I basically just have a wiki on our GitHub now. That's like, here's how you create the app. Here's how you test it. Here's how you do every little piece. I had to write it down because after the third time of relearning it, I was like, to stop. Right. Cause it's just a waste. And, it's too bad. Cause there, you would think the 900 pound gorillas would, have a slightly Experience for the people that are, building for them, essentially.
Brendan:Yeah. And it's also just coming back to rails, especially there's this idea behind the single person. Developer, developer experience, you don't necessarily need a team to build with Rails. And obviously you don't have a big team. And even with the team that I've got, I still find it. Well, I guess what I'm really getting at at the moment is by only having one developer or even maybe one or two, like if you're integrating with these really big clients and that's a ton of work, I totally see why you want it to be simple because you've got so many other things to focus on. But in a lot of ways, it feels like it's going against, not that Facebook is thinking about the Rails developer experience at all, but it also, it makes it harder to just think about as any company grows, keeping it to just one developer, whether you're working with Rails or not.
Ryan:Yeah, I mean, that, that is. I guess one of the big tradeoffs with integrations, they have network effects because, ideally you integrate with people with either tools that a lot of people use. or that is like critical to your workflow. Yeah. Right. You kind of have to have them it's a growth channel, if you're a marketing platform and you don't connect with Facebook or Google, it's sort of hard to compete. Yep. And it goes, I think you're right. It, it does sort of go against the rails, single developer paradigm. from afar, listening to other podcasts, like I know there's some people that seem like they're absolute rock stars with keeping, updates, for integrations going and good for them because it's not easy. I frankly don't know how they do it, but I see that it works and I see that they're doing it. So maybe I'm missing something. I don't know.
Brendan:Yeah, I think part of that has got to come down to the type of business. And I don't know all the intricacies of your business, but, just knowing that you've got a lot on your plate , I guess where I'm wondering is, Is it related somehow to the type of business they're running versus the type of business you're running?
Ryan:I mean, it wasn't meant to be a secret. I was thinking of Chris Oliver with Go Rails, he maintains what four or five different, probably more than that. I mean, four or five big gems. One of them is the pay gem. And that has an integration with Stripe. PayPal, there's another one in there. I forget it. And then that list is probably outdated from the last time I looked, but , he keeps everything up to date. He's on, he's on rails edge. He's on Ruby edge, he's on the most current version, of Stripe. And I know he's rewritten it a few times and he, and that's just one gem. And he's got like five other big ones that he maintains. his business model is do that stuff all day long. But that's still really impressive.
Brendan:Totally. I know , it makes me think of the 10 X developer, argument or whatever you want to call it, but yeah, not that that's, you know, we don't need to get into that, but it's impressive , I definitely struggle with. It's similar to where you're at where it's like, how do you just get everything done, and I know you're never gonna be able to keep up with that And I'm sure the crystals of the world are the same way like he probably has a huge list of stuff He's not getting done and he feels similar But he's not presenting that and obviously, you know who most people don't but I mean I as a manager like I'm always thinking Okay. Well, how are we gonna get all this stuff done? Because the backlog is so big and I know you're gotta be feeling the same thing but the part of it is you've got to You have to have that focus like we talked about earlier, I think that's the key and it's not easy, but just like keeping your focus on the things that at least at the moment are the most important and then hopefully, things work out, but.
Ryan:Yeah. I mean, it's as, you know, purely as a thinking as a developer, . I just want to go heads down and I want to build stuff. I don't care if it's a new feature. I don't care if it's a bug fix. I just, I just want to be coding. Right. And I, I don't, you know, you don't necessarily, when you're in flow, I'm not thinking like, all right, what is the impact of this on the business? Like I'm just flowing and I'm enjoying myself and I'm building something cool. And that's what. I wake up for, but on the other hand, our backlog is insane, ? And it's like, all right, how do you, you gotta manage the, you know, you got customer requests coming in and you got customer complaints coming in, you know, you got, requests from your sales team, your marketing team, your boss. and they all have, and I always joke with my team. They're like, Oh, I think this will be really easy. And I always just show them the picture of the staples easy button. Go ahead. Why don't you try it? Yes. Some things are quick. And some things are legitimately easy, but most of the time that is a gross oversimplification of the ask. And the question is, how do you balance that? Cause, obviously if you're an employee and your boss asks you to do something, you should probably do it. If you're, a business partner, your business partner asks you to do something, well, then maybe there's a conversation if you don't agree. And if it's a customer request or complaint, it's like, well, how valuable is that customer to you? If they decide to go a different direction, is that going to, right. You know, put you out of business, well, then that's a much higher priority than if, you can afford to lose. Now, I don't want to lose a customer,? But if they're not in alignment with values are, I'm not going to sweat it. And that, that lesson, I mean, it's still hard for me sometimes, but I've gotten way better at that over the last five to six years of just being like, all right, let's, you know, is this our, is this a priority customer. Is it, even if, like, I love the person and I want to continue working with them, if they have a different vision for what they're trying to do than what we're trying to build, then it's better for us to just say, Hey, we can't, we're not going to build that. Here's why. here's a potential workaround or if you shift your. Your process just a little bit, like this will work for you. or, here are a list of alternatives that, might be a better fit for you. And I'm happy to say that to people. And if they're, if I run into a customer that's rude or demanding or just like being unrealistic, I'm happy to be like. This isn't working, I don't want to work with you if you're going to be like that. we're both professionals. We can disagree and, that's fine. I'm not against disagreements, but, if you're going to make me miserable for working with you and providing, a valuable service, then no thanks.
Brendan:Yeah. Customer feedback is definitely a tough one because, you, as the product owner has, have a vision for where it should go and customer feedback obviously is very valuable, but when it diverges so much that they, maybe they love your app, but they really, really want that extra feature. And I guess to me that's the kind of thing where you, you file that away. And if other people ask for it, then obviously it's something you probably want to consider, but if it's just one person asking for it, unless they're, a whale. Size client that, like you said, might break the bank if they leave, then it's like, well, you kind of have to push back a little bit and, I'm not in the position to say one way or another, where I'm at. And so it's kind of interesting to hear you say, or you just talk about it a little bit, that you obviously have more say in what gets built, but still, you, you still have a lot of different things, pulling it, what you build, when you build things, all that.
Ryan:Well, that's my job, right. Is to decide, you know, take all the feedback. You know, we're getting, whether it's from our internal team,, our company vision, that my, my business partner sets for us, when we will kind of set it, but he leads it and, that I have the vision, how does the product fit into that vision? And I own the product vision. So it's like, all right, I know where we want to go to meet the company vision. And now let's gather feedback from the team, from customers. I generally follow the rule of, given that our backlog is. So big, but if I hear it once, I don't even really blink if I hear it twice, my eyebrows sort of goes up. It's like, all right, maybe there's something there. When I hear it a third time, it's like, all right, now I'm going to start actually exploring this. And then, I encourage my team to say, you know, I want you to be the advocate for the customer, if a customer, is making a feature request. I want you to be their advocate and come like argue with me on their behalf. I'm not going to say I'm going to agree with you. It's okay to disagree. I'm not going to just drop everything and be like, yeah, but I will try to give you a clear answer back. If they come to me with something that's like, all right, this is outside of our product vision, frankly, we're not going to build it ever. i'm just gonna tell the customer like, well, i'm gonna tell them that they're gonna tell the customer because , I think i'm a little bit too terse for customer support these days. Or I'll say, Hey, this is a great idea. It aligns with our customer vision and we're going to add it to our backlog. I have no idea when we're going to get to it, but we're going to keep track of it. And if more people request it and then, you know, maybe we'll build it someday. Or the third thing is , it's a really good idea. It is on my immediate roadmap. And I say, all right, I plan to work on this and. Two months and it should be done by the end of June and , we'll keep you posted. And over time those things coalesce, to a point where it's very rare that I hear a request that I haven't heard before at this point, I've been doing this company for 12 years. At this point, Either someone has already brought it up or we're talking about it anyway, cause it's a new trend in AI. We want to do something there, ? Like we have unlimited ideas and then it's like, all right, what's the most important thing right now? so it occasionally someone surprises me. They come up with a, what I think is a good idea that fits our vision that I hadn't thought of. But it, it's fairly rare at this point.
Brendan:Yeah, that makes sense. One question I have that maybe ties us all back to where we started is with customer requests that you get, do you ever have integrations that people want, with another app and that, that's where any of those ever come from?
Ryan:So the question is, are there feature requests for other integrations that we haven't built that I haven't thought of before? Yeah, things that you weren't
Brendan:planning and maybe somebody comes along and they're like, hey, It'd be great if your platform integrated with whatever it is, AI or I don't know, something like that.
Ryan:We don't get a ton of that. I think that the one that we have an integration with Facebook, we have an integration with Google business profile, which formerly Google my business, like those are the two big ones. We plan to do an integration with LinkedIn and Instagram, and those are the four that, you know, LinkedIn and Instagram are the two we get most requests for. So those are definitely on the roadmap. The other one we get from time to time is integration with MLS. We have, a handful of, real estate agents or realtors that, use our platform and they want to pull in, MLS data, to show properties. And that is something we're probably, I won't say never, but it's, it's pretty low on the priority list because there's tons of tools that you could just drop in as an embed. And, that's a complex. Um, so that's, that's one that I get that we usually say no to, we don't get, most of the things that people want to do can just be dropped in as like a widget, and maybe that's just based on, the size, of our company and the type of client we currently have that is just haven't gotten more of those requests, but I'm open to other ideas, I just haven't seen them yet.
Brendan:Interesting, cool, should we wrap it up?
Ryan:Yeah, let's do it.
Brendan:All right. Sounds good.
People on this episode
Podcasts we love
Check out these other fine podcasts recommended by us, not an algorithm.
Code with Jason
Jason Swett
Standing in the Fire
Very Good Software
Remote Ruby
Chris Oliver, Andrew Mason, David Hill
IndieRails
Jess Brown & Jeremy Smith
Startups For the Rest of Us
Rob Walling
Vibing with AI Code
Brendan Buckingham