Rails Business
Brendan Buckingham and Ryan Frisch talk about developing with Ruby on Rails and how to leverage it to build a business.
Rails Business
Mateus Pereira
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
In this episode, we chat with Mateus Pereira from Ombu Labs about the nuances of Rails upgrades and how AI is revolutionizing the process. From tackling gem dependencies to client collaboration, Mateus shares valuable insights and tips for executing seamless upgrades. Perfect for developers looking to streamline their Rails projects and leverage AI tools effectively.
00:00 Introduction and Casual Chat
00:25 Meet Mateus: Senior Rails Engineer
01:29 Mateus' Journey into Programming
02:17 Joining Ombu Labs: Trials and Triumphs
03:33 Lessons from Rejection and Persistence
04:35 Challenges in Rails Upgrades
08:12 Tackling Gem Dependencies
20:04 The Role of AI in Rails Upgrades
30:47 Discussing Ruby Critic and Rails Bump
32:07 AI in Code Refactoring
33:58 Client Relationships and Project Management
44:14 Challenges with Test Coverage
45:32 Manual Testing and QA Processes
49:18 Using AI for Writing Tests
55:37 Final Thoughts and Sign-Off
LINKS
Questions or comments, email us at railsbusinesspod@gmail.com
Hey man. How's it going?
Brendan:Good. How you doing?
Ryan:Yeah, not too bad. Another, another day here at the office, and I think we're all fighting some sort of cold today, so let's see if we can power through this thing.
Brendan:It is November.
Ryan:yeah, well, yeah, November back to school,
Brendan:Yep.
Ryan:weather change, all of that fun stuff for me anyway. Um, all right. Well, today we, we've got Mattias and we were joking beforehand 'cause I, uh, am unable to, uh, pronounce his name. Um, but Mattias is a, uh, senior Rails engineer at OmbuLabs. Welcome to the podcast, and maybe you can, you know, tell us, tell us your full name since I couldn't figure this out.
Mateus:Well, um, it's a pleasure to be here. Uh, yeah. So if you want to be super correct and pronounce it as it is in Portuguese, it would be Matos. Um, but usually Americans just pronounce it as you. said it right now. Um,
Ryan:apologies I've been, I think, you know, full disclosure, I've been working with you for a while now and this is the first time I, you, you told me that I was completely butchering your name, so apologies.
Mateus:oh yeah. Um, and again, like I said before the show started, I already have like no expectations concerning that. And I'm already used to people calling me like Matias or Mattias and you know, that that's just how my name sounds in when Americans say it, I guess. Um. But yeah. Uh, so I guess just to a little bit of my background, um, I started working with Rails. mean, not, I wouldn't say too long ago. It was in 2018. My background is actually in physics. Um, and, you know, due to various reasons related to, um, not being really interested in academic research and all that, I decided to start a programming. And out it, you know, it, I guess it was the right choice. I really like it. I really have fun with it. I started working at, um, what would be the equivalent of the DMV here where I live. And I spent like three years there. Then a short, um, let's say interlude with a Brazilian startup. And that, that was like six months. And then I got to BLE Labs just because at the time would. mean, I still do listen to a lot of podcasts. Right. Like res business or other stuff. I, this curiously, was during, um, Jason Swett's PO podcast. I heard his interview with, um, Ernesto, and I was immediately interested in the company and I was like, Hmm. Interesting. Let me check it out. Saw they had ads for jobs and I believe the first time applied, I was rejected actually. And, you know, um, but that, because that's, because u usually at BLE labs, it just so happens due to the nature of the work, since we, since clients usually come at us with, you know, rails, upgrades and things like that, it's very, it's a very hard demand for, for like, um, a person who's. Starting working on rails and all that. Um, but yeah, so, you know, I just let it be and waited, I guess a few more years. at some point I decided to apply again and this time I was able to get in and I've been there since. It's gonna be five years now actually.
Ryan:Yeah, that's a really cool story. And I think a lot of people would, you know, oh, I got rejected. You know, that company stinks. I'm not going back to them. And they, they give up on that company and they move on. But, you know, a good lesson here for, you know, anyone who's just getting started is. There's a lot of reasons you could be rejected, and of them may have nothing to do with you or your experience, and you don't know. Right. So don't, you know, don't be afraid to reapply. And I, I know 37 Signal says that all the time, right? They get, you know, thousands of applicants and, you know, if you listen to the Rework podcast at all, which is another good one, and I think this is the second week in a row, we've, we've mentioned this on the, on the podcast, but you know, they. of the, the stories they tell are people that, you know, got rejected 2, 3, 4 times before they got hired. So I, I, I think that's just a, a good reminder and a cool story. Um, and of course I'm happy that, uh, now working for OmbuLabs 'cause you helped, uh, my company upgrade our, our Rails app, uh, over the last what year and a half or so. Um, you know, gi, given that your primary focus has been. Around Rails upgrades. Um, you know, maybe you could share some of the, you know, the gotchas you see as you're tackling a rails upgrade. You know, where, where are the spots that are most common for people, that people get stuck, um, or that you can help unstick them.
Mateus:Yeah. Uh, sure. I think, um, funny, it, it's a question that I feel like I'll, I'll get to it soon, but I feel like it's, it's funny enough, ironically, I guess it's a bit hard to answer from my perspective because usually, um. I think the reason companies come to us looking for rails, upgrades and things like that is because they just cannot find time, or they can't organize themselves enough for various reasons. Right. I'm not, I'm not saying that, you know, it's all just companies that aren't organized or anyway, or anything like that. It's just that they just have so, so many developer hours on their books, right? And they need to do features, they need to do a lot of things. And sometimes the time they have left is just not enough to keep up with their tech debt as time goes on. And so they come to us. Um, I feel like, and you know, maybe this is answering more question of like, Hey, I am in. Uh, I guess I'm the owner of a company or whatever. Um, how do I get unstuck? And this might not be super helpful. More like, yeah, to get unstuck, just hire us. Um, but.
Ryan:I, I mean, I, I think that's part of it, right For me, I tried to upgrade our rails apps. Several times. Right. And I, I think there's a, you know, and I've been following the, the FastRuby blog for years. I've read the Upgrade Guide, which, you know, I found helpful. I still got stuck, right? And it usually was around, uh, GEM dependency upgrades.
Mateus:Yeah. No.
Ryan:know, 'cause, you know, changing the, the line in the file to upgrade Ruby or to change it to, you know, the next version of Rails and even to dual boot, which is something I know you guys talk about a lot. you know, so that you're running both the current version and the next version. Like that's, that stuff was. I don't wanna say easy, but it wasn't like difficult, right? Like you, you could sort of muddle through it. But then I'd run like, you know, I'd try to run bundle and it would be, it would fail, right? And then I'd be like, all right, well let's try to figure out why it's failing. And I'd be like, oh, this gem seems outdated. And then, you know, to upgrade that one. And all of a sudden, you know, 10 other gems are now, you know, have a conflict. And I couldn't. I couldn't get past that and, you know, bear with me here.'cause I'm, I'm, I'm coming to a question, the, you know, I, I, I researched, I listened to people and at one point it was like, all right, I need to, you know, the, the thing that I tried was like, I'm gonna lock every version of the gem so I know exactly what I'm getting. And that, that was the approach I took before I ended up coming to you guys. Um, and that didn't work either. Then as I watched you guys work, from my perspective, it, it seemed easy for you. Right? So how so, so, so the question is like, how did you, how do you approach gem dependencies, uh, during the upgrade process, right? Maybe that's a, a good place to, to dive in.'cause I, I don't think I'm the only person that has that.
Brendan:I, I just wanna note for those who can't see us, that as soon as you said it looked easy, Mathias, and of course I'm butchering your name now too, but laughed so clearly you have a good answer for this.
Mateus:Yeah. I mean that's why I still think it's a matter of, um, a, a lot of having the availability to keep trying, because it, I'll put it this way, um. What I think the thing that helps us the most is indeed that number one, since, since we've been doing upgrades for so long, we already have this whole, how can I put this knowledge store on how to fix issues, right? So many times when we see like a gem that's. Just refusing to upgrade because, you know, whatever version doesn't work in this specific jump. We, it's usually a gym that we already know. Like, oh, alright, so this gym has this issue. We need to go to that specific version and that one will bundle. Um, there's that. Right. And so when I started, yeah, I was pretty much in the same position when I got my first job upgrading, um, an application, I think. I'm pretty sure, well, not pretty sure. I'm certain it did not really go well with the client. Yeah. They, they weren't like, it's not that it didn't work out, it's just that they clearly were expecting more. Right. But that was because of my inexperience. So, you know, that was a thing. And as I progressed, as I got more projects, I was, let's say, wiser to, you know. lean more on my colleagues and ask for questions and, and, and because there were people at, there are people at OmbuLabs that have been doing this for longer than I have, have way more experience they just are able to point me in the right direction. Now, think that thing that might, uh, I guess in, in practical terms help for specifically for when you're trying to update, um, your GEM versions. Is, and it might sound, um, I don't even know the, the, the term in English, but to me what clicked for me was to slow take things slow, slow as in very, very patiently. Um, because for one error messages for these things are just not very good at all. They'll list you sometimes they'll list you a ton of things that have no issue, no compatibility issue whatsoever, and. You're just like, okay, what, what's, what's going on? Well, what is the problem? Which gem of these does not work? And you start getting desperate, right? You start thinking, Hmm, no, no, no, I was trying to upgrade this one. Let me just try a different version.'cause then it may, may, maybe it works or you try to change a version of one of their dependencies and maybe it works. And usually, um, I've, and having gone through that, after I just stopped to decipher all of those messages, which is very annoying, but it is possible. Um, you'll usually find that one little gem hidden in the middle of all that red text that is in compatible. Then you just like, oh, I just need to add this one to the bundle upgrade, and then it works. So I think, and again. I know it is a very, um, can I put this? A very, uh, uh, of, of, of advice that's saying like, yeah, yeah, you just need to try harder. Yeah. But, but I don't, um, at least I don't mean it that way. I just think that maybe it's, it's a matter of a certain mindset res upgrades, they are very annoying. And I've done like, or better yet, let me put it this way. I did rails upgrades before envelopes. It wasn't my old job, and it was one of the, uh, worst experiences of my life. I had to upgrade apps. They were apps that were in four and five and had to go to six. Right. And from five to six is, is one of the biggest, um, it's one of the jumps that has the, the most amount of changes. And let me tell you, I thought I would never crawl out of that. Hell. It, especially related to, um, the web packer changes, right? The, the, I even, I even forget what specifically, but the, there were a lot of changes related to the Rails asset pipelines and, you know, the way things were done in five just didn't work with six anymore. And what was worse was that we had just migrated things in five to, you know, use web packer and things like that. And then in real six. The d hh and slash rails team was like, you know what, let's just go back to what we were doing before. and yeah, it was, it was horrible. But having done a ton of rails upgrades, including five, two to six, oh, at this point, it really was a matter of just, if I'm gonna do this, I really need to focus on it. And. Take it slow and not slow as in, you know, actually slow as, but more as in calmly read through everything and understand the changes. And then, um, as daunting as the changes may seem, which there are many, be put it in your head that possible you'll get there, but you'll.
Ryan:Clearly people have upgraded. So I That is, that is, uh, encouraging, I guess. Well, so let me ask you this, when you are, you know, those. You know, messages, Are you going, you know, to the gems for, you know, for each gem and looking at the change log and trying to understand, you know, breaking changes versus non breaking changes. Are you opening the GEM and the editor and, and looking at the gem's code, right? Like, you know, you, you said at the, you know, somewhere in the middle of that. Last piece you said. All right, well, you finally identify, you know, the gem that needs to be upgraded and, you know, maybe it's one of the dependencies of another gem, you know, what does that actually look like in practice? And I, no, I think I know the answer to that, but are, are you just, you know, taking that gem dependency and explicitly defining it in the gem file so that it sort of locks it to the version that the, the other gem needs? Like how do you actually resolve it at the end of the day? So I guess there are like three questions there for you.
Mateus:Uh, and in fact, um, thank you.'cause like, uh, um, uh, again, just bringing back about a bit about the last answer I gave. Um, I think this, I think what I needed is, was to think in terms of, of, in more concrete terms, right? Um, but yeah, uh, usually for upbringing gems, what happens is. Well, you try to bundle with the new versions, right? They don't usually work from the get go. And then go through, um, um, the error messages and then you, after sifting through the ones that are just fine, but are printed there as an error for some reason. You find the one that actually, oh, this one is incompatible with, um, with, uh, this version of frails. It is usually, a matter of opening up the gems repo. to exchange log. Hopefully it has one they usually do. and then trying to, you can grip or just read through it, um, and try to find, oh, added Rails x um, support. And then it just gives you the version. And at that point, it's a matter of just flipping the version and paying attention to whatever it might list as being breaking changes or API changes. Um. You really, the ones that are better are obviously the ones that just like change the rails, restriction in their gem spec, right? Those are just excellent. Um, but of course there are the. Sorry about that. Uh, I hope the wind isn't getting too much in the way. Um, but, uh, there are those that will require you to do API changes. Now those are usually the, let's say, more annoying ones. Um, and there are examples like, um, I think AWS is one that every now and then changes something. Uh, although I, I think it's not too bad. Uh, because they usually just change like some, you know, config or whatever. Uh, and yeah, that's, if you have this, um, little process of just the habit actually of checking Gemfile repos and reading their change logs and read mes, it makes things actually quite, um, smooth. To be honest. So I guess that's it, right? That, and, and I guess this goes into the part of why saying but about being patient like, 'cause I remember myself doing this and seeing these errors, I would just, would not have the patience to like open up the gems repo and read their, read me and read their change log. And when I did it, it was quote unquote.
Ryan:My recollection of my experience was like, I, you know, I attempted that workflow that you just said. Right? And I'd be like, it just seemed like it was a rabbit hole that never ended sometimes, where it was like, all right, I know, you know, like, uh, I'm trying to think of a good example from, from our upgrades, but I can't think of one off the top of my head. But like, I'd go and I, I'd have to upgrade. Let's see, like Sidekiq. Sidekiq in Redis is like one that haven't actually upgraded that one, uh, to where I want it to be, but you know, upgrading. Redis required a certain version of Sidekiq, and Sidekiq required a certain version of Ruby, and Ruby is supported by a certain version of Rails, and that was just at the top layer. And then once you get into the dependencies, you know, you, you get a, a whole nother slew. So like, I feel like I'd solved one problem and the next problem would pop up and maybe I just didn't. Stick with it long enough to get all the way through the tree. Or maybe there was actually, you know, another blocker and you know, I needed to, you know, maybe if I tried it again now that we're on rail seven two, it wouldn't be as big a deal as when I was on Rails five or rail six. Um. But anyway, um, all right. So we, we had Ernesto on, um, you know, he was one of our early guests on the podcast when we were just getting started, and I know at the time he also shared, you know, information, uh, about your sort of internal repository. I mean, nothing, you know, specific, but just that you had it. And then we also got into the fact that, you know, you know, AI was. New-ish. Right. I feel like was probably six or seven months ago and a lot has changed, and know you guys are now working, you know, you have a bunch of AI tools that you put out there. So my question to you is, how has AI sort of changed the process for you specifically in terms of, of upgrades, right? Uh, has it made it easier? Has it made it harder? You know, how are you leveraging it, if at all?
Mateus:Yeah. Um, the short answer is, um, so much easier. Um, yeah. Uh. I think, um, specifically for us, the way we leverage AI varies a lot. It depends a lot on the client. So like for, for, for for Lable, right? Um, it's easier because there aren't as many restrictions on the tools we can use with the contract we have with you, with you guys. Um, but for example, when they hire us for a rails upgrade, which is usually a bigger company. Um, and by big I mean like actually big companies that have a whole, uh, you know, legal department and a legal department for their legal department or wherever and everything has to, you know, go through contracts that have to have like five signatures on them. Um, then yeah, they have a lot of restrictions in terms of how, in terms of how we can use the data. And so I can't just like, you know. Open up their project and plug in Cloud code or Codex and be like, Hey, what's wrong here? Um, yeah. Uh, but even for those, usually the companies now are all adopting AI in some way. So we actually had a client um, would actually give us licenses to, um, cursor, right? I was able to use AI that way. Um, others, they just won't have it. And, you know, we'll have to be, uh, very, very, uh, circumspect with how we use it. Right? So, like, we already have the whole experience thing, it's usually, I end up usually using it for nasty little bugs whenever we've, you know, we've updated the gems where we have the, the test suite running in the next version, and now we have to fix them, right? And then, you know, we start fixing a whole slew of them that are easy fixes that are just like, oh yeah, this, this method name changes in rails, or you need to change this config or something or other. And then there are always the ones that are actually issues on, you know, how the application was built and now rail's changed something, or God forbid, the, the app is very, um, creative with internal API. Uh, and you know that rails will, will really change and not even, um, let you know what happened. And, you know, then you start debugging and you have to figure it out now. And for those I usually try to fall back, say when I find those.'cause it's, I find that it's pretty in general at, you know, pinpointing problem. Uh, and that's how I leverage it, right? But then I just have to have this care really, um, when, when doing this related to data. So, you know, for Lable I can just pretty much use an AI tool for bigger clients. I have to, know, think of the general problem that I'm trying to solve, that I'm seeing, and, you know, I explain that the LLM and then I, an answer and try to go from there, right? That's kind of how I do it.
Brendan:I, I've got a.
Ryan:just, just to clarify, when you say data, you don't mean like customer data, you
Mateus:No.
Ryan:code?
Mateus:Yeah, no, no. Yeah. When I say data, I mean the, the, you know, their, their code or anything associated with the project that they gave us access to. Right. Because even,
Ryan:just wanted to clarify that even for local, any, any lable customers, like I'm not giving him to our customer data, right? Having access to the code is a different thing. Um,
Mateus:Yeah.
Ryan:altogether,
Mateus:Sorry, maybe. Maybe I used the wrong word there.
Ryan:what were you gonna say, Brendan?
Brendan:Yeah, I was just wondering, just maybe diving a little more specific into how you're using ai. So if we go all the way back to that example of upgrading just a gem even, and you've got the, it's not really a stack trace, but it's kinda like, like you said, it's like that tree of all the different things that it depends on and maybe the thing that you need to actually. Change is the one right in the middle, but you don't know that. Have you found even throwing something like that into ai, does it help? Does, does it actually have enough context to know that, oh, well, in the majority of cases, when you're upgrading, we'll just use the Sidekiq and Redis one that Ryan brought up, you know, Sidekiq. And this is all made up to, you know, it's gonna be this. Middle dependency, whatever it might be. I don't even know. Some, some middleware thing like, does that make sense? Is that something that you are finding value or is that not exactly what you were getting at before?
Mateus:Yeah. You know, actually, um, it, it does make sense. Although, funny enough, I've never tried it because I just got so used to like reading
Brendan:Free. Free advice. There you go.
Mateus:Yeah, that, um, that I just never thought, you know what, I can just put this Sarah message in, into I'm, and ask for it. Just tell me which line there is telling me something is incompatible. Um, uh, but yeah, and I never tried. I don't know exactly how it would respond. I, I would expect it to be able to figure it out,
Brendan:Yep.
Mateus:but, um, yeah.
Ryan:think it's largely dependent in my experience on whether the gem has a change log,
Brendan:Yeah,
Ryan:right? If it has a change log, you can basically tell it to go look at the change log and find
Brendan:I.
Ryan:the braking changes are for your version of Rails and Ruby, and that has. Help me when it's a more straightforward gem upgrade. I, you know, the more complicated ones, I, I don't recall if I tried that or not, but I would imagine it's effective with the well-known gems.
Mateus:Yeah. Um, I think 'cause I think maybe, so maybe because I approach this in a very, 'cause I'll put it this way, I'm, although I use AI a lot, I am very wary things it tells me, wary as in I'm afraid to give it too general of information. I try to be super specific. then I usually get better results. what you're describing there about like, oh, it depends if it has a change log that you can point it to and all that. Maybe going off on a tangent, because I'm actually testing this out right now. There was this article I read the other day. This fellow actually wrote a skill that does rails upgrades.
Brendan:Hmm.
Ryan:Hmm.
Mateus:And it, it is like, it's the full package. Well, at least it tries to be. I'm testing it out to see like, you know, how it handles like big old repositories. but um, it, it quite literally just has all of those descriptions. Like, yeah, check if the gem isn't compatible, check the change log. What does it say, what version you need to go, what are the breaking changes? Blah, blah, blah. Everything. It does precisely what you just. Said, um, and I, I tested it out in, um, I want to remember it was, it was, um, it was some open source, um, CRM app that I found, and at least apparently it went well. But I didn't really, you know, run the app and try it all, but it identified the things that I expected it to find. And just changed it and did everything. Now what I did was I tried finding like big repositories in GitHub. So I'm testing out with one cult forum and I will test it out with GitLab and see
Brendan:Sure.
Mateus:'cause then you know, you're starting to get like,
Ryan:On. We gotta get, gotta the first of, we can give that.
Mateus:yeah.
Brendan:Well, one other thing I just wanna follow up with really quick, 'cause I was also thinking of this when you were talking just about AI and also your, like all the experience that OmbuLabs has. To me, maybe you're already doing this, but it, it feels like there's possibly an opportunity where if the LLM, like with the question I asked a minute a couple minutes ago, if it can't handle it with, you know, just straight outta the box CLO or Codex, you know, if there's a way that you as OmbuLabs can somehow, like if you have like a, a. Knowledge base, right, of like, Hey, I've seen this gym always conflicts with this one or whatever during your upgrade from four to five, whatever the example might be. I'm just kind of curious if that's something you guys have even explored around, like feeding in your own knowledge to help AI kind of figure out, okay, we've seen this one before. Um, I don't know if that would be, you know, what that would be necessarily, if it's like a, a rag or a MCP or what it might be, but I dunno if you have any thoughts on that.
Mateus:Yeah. Um, so, you know, and in fact, um, maybe, maybe, um. Uh, an indication for a future guest you should call on Amanda, who's our, uh, chief Operations everything person. Um, she's the one who's actually a specialist with, you know, artificial intelligence and LLMs and ml, or, you know, maybe she can correct me in in her. her resume. But, my point being the, the, the internal team I'm part of at EPS is precisely dedicated to investigate these things because we are trying to, um, figure out, hey, how can we leverage AI everything that we know and have collected about, you know, gem and compatibilities. Uh, the little hidden quote unquote gems that, you know, these, these, these libraries have, uh, of changes that they just forget to report. And, you know, we have to figure it out through debugging and we just have it annotated somewhere on how, you know, how you do this change and whatnot. How can we combine those two to, you know, really make, uh, our upgrades, uh, go faster?
Brendan:Mm-hmm. Yep.
Ryan:And I think it would be really interesting 'cause you guys, uh, I know, um, Ernesto's sort of been on a kick of collecting, uh, developer tools, uh, open source developer tools, you've got, you know, um, shoot, now I'm blanking on it, but reek and,
Brendan:Oh, uh,
Ryan:all, all the, all the tools that, do you know what I'm talking
Mateus:Yeah.
Brendan:yeah, I can't think of the names, but I know what you're talking about too. Yeah.
Ryan:yep.
Brendan:Yeah. Ruby Critic, that's what I was trying to think of. Yeah.
Ryan:So like, it
Brendan:I.
Ryan:to take it a step further, right? Like you guys have a lot of, and you just have the, the ruby bump gem too, right? Like that you just took that over
Mateus:Yeah.
Ryan:one that
Mateus:You mean the rails bump site, right?
Ryan:Rails bump Yes. Are not ruby bump. So like you get a lot of data that not only helps with upgrades, but it seems could just help with general. Maintenance tech debt refactoring. It'd be really interesting if we could get to a point where it's like, all right, you could take all that and like feed it in. And it's like, all right, well, you know, here are the most complex. Areas of your app and here are, you know, following sort of the common rails patterns. It was like, all right, if we, we can extract this object and, you know, this decorator and this, you know, form, presenter and, and, and, and start to like thick, you know, I look at Ruby critic for example, and it's like, all right, I know that here are the three controllers or the three models that are just, you know, too complex to work with. But how it's hard to change those things, right? That's why they're complex and they, they, they've gotten to a point where the complexity prevents change, but, you know, you're sort of a chicken and an egg situation. But it, I've started to experiment with using AI to Some of those improvements, right? Like, you know, here's the problem, let's focus on this one. You know, sometimes it's just this one method in a model. help me come up with a plan to, you know, move this to a porro, um, you know, just a plain old ruby object or, you know, how else can I approach this and break it down and help me break it down into. You know, a migration plan, um, not a data migration, just a refactor migration. but it'd be interesting to connect all those dots, right?'cause the upgrades are one form of tech debt. But I feel like as you're upgrading. You see so many opportunities to improve the maintainability of your app. And you know, I'm not suggesting that you just refactor for no purpose, right? But if it's refactor to make the app easier to work with, I'm generally on board with it. Um, but I feel like AI could be a huge help there too. Um,
Mateus:Y Yeah, no, um, definitely we're, we're trying to get to that. Um, and mainly because it, well, obviously would help us as a company, right? Because it it if, if we happen to be able to build a tool that can incorporate all, all this data and use these LMS as a way to, you know, upgrades, even plan refactor for clients and things like that, then you know, that just the way I see it at least is that it would just immensely increase the bandwidth we have to like deal with more. simultaneous projects and things like that. not to mention that, you know, um, well, you know what I'm gonna hold, I'm gonna hold my tongue because.
Ryan:Okay. well you brought something up that I wanted to, to touch on, um, which is, you know, obviously, you know, fast, Ruby sort of has the agency model or, you know, uh, s you know, it's not software as a service, but it's like service as a service. Um, in any case, you, you work with a lot of. Clients simultaneously. Right. I'm sure you spread that out across your team. I think a lot from, if, if you were to flip it around and look, you know, from the client's point of view, obviously they're, you know, most people are trying to accomplish something like an upgrade, but do it as efficiently as possible, right. Both in terms of time and money, um, you know. They'd much rather probably be doing things that, you know, focus on features and, and driving value. not to say that, you know, upgrades don't have their value because not being on old versions with security issues is, is valuable, but I, I think it's just a harder sell. so the question is like, as you're working, you know, with clients, like what do you wish they knew? Or how do you, you know. there anything you wish they would do? Like just general advice to make the relationship and the process flow better? because I always, I, I look at it from my side, you know, as the client and be like, you know, I, I always wanna make, I, I always want to have communication. I always want it to be as efficient as possible, but then I try to put myself in your shoes and it's like, all right, well they're running a business, right? They have different constraints. Is there anything you can share from your side that would help people on my side be like, all right, this is how to think about the relationship and how to get the most out of it. Uh, and maybe that's a, a very deep question, but I feel like that maybe we, we, we pull it to the, the business side of things a little bit, and if it's not, it may not even be a code answer. Um, I don't know if you have an opinion on that.
Mateus:Yeah. Um. I mean, I don't, I'll say this, I don't have a very strong opinion on it, but to answer your question, if there's something I wish clients knew when they come to us is, and this doesn't apply to, um, all clients, uh, sometimes, and, uh, I wanna preface this saying it is also fine that it works this way. We actually clients to behave in many different ways, and we already have, like, you know Yeah. If the client is like this. Then, you know, act in this manner. Right? Um, but my general advice is, I feel like it usually, the, the projects that go smoother are those where the client can dedicate at least some time to be a bit closer to it. Because the thing is, especially for upgrade products where, you know, the apps are bigger. more complex. They have code that's been written when Rails maybe was pre version one hanging about. Um, and sometimes they just wanna like, you know, here's the app. want to get to this version. Just, you know, ring us when we're done. and it's, those are usually the hardest because. Many times you're gonna hit wall that is super specific, right? You need to know something of the business logic. You need to know how that code was written, why it was written, and then you start, um, if the client is more of this, you know, hands off approach, like, here's the job. I really want to not be bothered with this 'cause I need to do other stuff, then, you know, you're gonna have to start sifting through, um, git histories and all that kind of stuff. So, you know. Wow. Again, it's not something that I feel like, you know, oh, if a client is like this, like that, then you know, it's, it's not a good client. I don't think that it's just that, hey, you're able to be a, just somewhat hands-on and available to, you know, uh, answer questions and open to even be a bit proactive, more, oh man. I'll say this, for example, the relationship with Loomo, right? You usually like. To, you know, for us to tell you, Hey, we stopped here and you wanna know the things that you can probably move forward faster just because you have more context into that. I feel like that is, is the relationship that me personally as a developer, I think is the best when doing these kind, this kind of work. but again, uh, also understand that from the client's perspective, sometimes they really, really. Need someone to take it over and they just don't have the bandwidth to, to be paying attention. Right. And you know, we're there for that too. We can do it too.
Ryan:Yeah, I, I think the thing that I'm hearing is that you sort of have different personas, right? You, you, you realize that there's different types of people that come to you and, you know, perhaps I'm maybe putting words in your mouth, but you can treat them differently, right? Uh, to meet their needs. Um, but for, for the, the folks that able to. Dedicate a little bit of, of resources to help with the project. Right. You mentioned, know, communication. You mentioned like, you know, hands-on, you know, is there anything in particular that, you know, you wish they, you know, how, how they were interact? Like, I, I always think, you know, using our, you know, our work together as an example, I want to be a resource. For domain knowledge, right? Like, I can look at a certain part of the app and say, all right, you know, this is important or this isn't important, or we can just ignore this because like, I know we're gonna deprecate that as soon as this branch is done and you know, I don't really care if it works or not. Um, or, you know, this part of the code base is dead, like don't worry about it. you know, but I, I also worry at the same time. I don't want to get in the way. Right. I hired you to do a job. Right. and I want to be a resource, but I don't want to be, I don't wanna micromanage either. And that's a really hard balance for me. And, and not just working with you, like working with any contractor, right. Like I. don't actually wanna work with somebody who isn't gonna try to think for themselves.
Mateus:Yeah.
Ryan:right. So, like, I don't know if there's any other advice you'd give for people that feel the same way as like, you know, would you rather over communication you know, to, to work in a specific way in that regard.
Mateus:Yeah, I, you know, I think, I think that the, the sensible, the, let's say the, the sensitive thing about this topic is just that, like I said, right, depending on, on the bandwidth of the client, their size, right, how they can deal with this. You know, whatever answer I give might, you know, be kind of like, yeah, but I just can't do that. Uh, but to, your question, I I do feel like, for example, you strike a good balance, right? It's, it's neither too intrusive it is precisely what you're saying, a source of domain knowledge for when we need it. And, you know, I guess that is precisely the trick and know it's kind of like, yeah, alright, but how do you do it? That that part I, it's really hard for me to, to say, yeah, no, just make sure you communicate this way or whatever. Because each client has their own processes and their own way of speaking and their own expectations. Um, but I guess what I can end the note, I can end with specifically for this, this question is. I've rarely encountered a client that was actually, for example, annoyed that we asked them questions. Yeah. Yeah. Th th those, can't even recall one that was like that. Um, but there are those that are way more kind of like, um, yeah, there are certain questions. have to be'cause they might feel like, wait, I'm hiring you precisely to, you know. Figure these things out, it's just, you have to be, um, you really have to, let's say, use your intuition, I suppose, your, your, your skill, your, let's say your interpersonal skills, right? You talk with the developers, you talk with the person who's responsible for it, and you can kind of see what they're expecting. And Yeah. Uh, but in general, of course, if, if, if the client is more open to just be available for any domain level question, uh, or very specific code that was written, God knows how long time ago, and nobody knows anything about it, or at least we can't just fathom why it's there or what it's doing, that obviously makes it easier, of
Brendan:I would've to think too, that like. A better test suite might help with some of that because at least if there's a test suite that maybe is covering that code you just talked about, that you have no idea why it's there and it's causing a problem. Like you, I, you, you correct me if I'm wrong, you might have at least a little more confidence in, okay, if I change this, I can at least know it's broken. But obviously, I mean, we don't have a great test suite, so like I'm not a great candidate for candidate for that myself, but. That's also something I was thinking about as you were talking and answering these questions is just like during a rails upgrade, how do you know that you're not breaking stuff beyond automated testing? Right? Like you could say, okay, I did everything to check off all the boxes that all the gyms are now working and all the things we've been talking about, but how can you prove that the business logic of the app is not broken? You know, that that's, that's gotta be a tough thing to do.
Mateus:Yeah, it and, and it is. Um, so the thing is, fact, before, uh, when I started BLEs, we, we were actually quite strict with, um, code coverage. Um, requirements.
Brendan:Yeah, I remember some of those conversations with Ernesto and like podcasts I heard and stuff. Yeah.
Mateus:yeah. Precisely for this reason. Um, because like if, if you had, if you come at us with 30% code coverage, I mean, we know nothing about the app. We can go on changing things, but. do we know that we're not breaking stuff? And over, as the years went by, we kind of found ways to, let's to add a bit of wiggle room there, depending on the contract, depending on the side of the app. And you know, what we usually do is, hey, if you have less than X code coverage, I think, I think it's still like 70%. We'll always preface things with we can do the upgrade. We'll do the changes and we will, you know, make sure that the tests you have are green. However, it will necessarily, uh, you'll necessarily need to do some form of automated manual, but you know, some form of QA make sure the app is running because it's just not possible for us to know. Right.
Brendan:Right.
Mateus:it's literally impossible.
Ryan:we had. We definitely have less, we're locs under 70% test coverage. Uh, you know, something that I'm working on improving. but our process was they would get the app to boot and deal with any, uh, tests that were failing. And then I had a, basically a test script that I had written out that said, all right, I need to go through this. Every time they do an upgrade. Now will say. You know that going through that script would take me a day or two of actual time. Like it was not a short script given the size
Brendan:Just, just to be clear, are you talking about like your script of you manually do taking steps?
Ryan:this is literally a Word doc, and I
Brendan:Yep. That's what I figured.
Ryan:that I
Brendan:I have something similar. That's why I wanted to make sure we're thinking on the same thing here. Yeah.
Ryan:And I put that doc together initially because all right, here's my roadmap for the test that I need to
Brendan:Right. Yep. Exactly.
Ryan:Like if I had tests for all of this stuff. I mean, I would save, I, I probably would've saved months of effort,
Brendan:Yep.
Ryan:Um, you know, time. you know, so that's what I do. I literally have that dock. It sits on, you know, my desk or next to my desk. And as I am writing code, I try to add tests for the places that as I go, as time permits. But otherwise, I, you know, every time we got to the next sort of upgrade. You know, you and the team would be like, all right, it's ready for qa.
Brendan:Yep. Check it out.
Ryan:two or three days and I'd go through it. And then I'd say a lot of cases, I'm also a developer and not, you know, just a, you know, business person, if I ran into something small, I'd fix it myself as I go because it would just be more efficient that way than to, a lot of times it would take longer for me to explain it or shoot a video or put the ticket together than it would be to just commit a thing. Um. only downside with that, other than it takes me a lot of time, is I wasn't gonna do that multiple times throughout, you know, the upgrade. So if we were going from Rails four, two to five or whatever, right? I would wait until they were like, all right, everything is done, I think,
Brendan:Yep.
Ryan:Like, um, it was just not feasible to do it multiple times per rails version. and that and that. It was not the most ideal process. I'd rather do it more frequently, but that's just the, the reality of,
Brendan:I'm in the same boat. I totally get it. Yeah. Like it, it's always. Because we've done some rails upgrades as well, not, not with OmbuLabs, but with a lot of consultants too. And it, and even with, it's not, when it's not with a consultant, it's always just this, if you don't have a great test suite, and even if you do, you still wanna probably do some manual poking around, but it is a very intense process when you don't have a test suite you can trust. So, yeah, I, I'm just thinking about that the whole time. It's like, because you're an outside consultant, you don't know the business logic. In and out, of course,
Ryan:even know what it's supposed to
Brendan:Y.
Ryan:right? Like, I mean, you, you can read the code and you still might not. Make the right inference of Right. This, this was the point of this, part of the code base, right? Like
Brendan:Yep.
Ryan:code can be misleading as can tests, right?
Brendan:Exactly. Yep.
Ryan:because it's, it doesn't have to, just because it's, it exists a test exists, doesn't mean it's a useful one.
Brendan:Right.
Ryan:Um, which I find more often than not. Uh, unfortunately. But anyway, I, I think we've been talking for a while. I think that's probably. good place to, to wrap up any other, thoughts, last questions.
Brendan:I'm good.
Mateus:Yeah, no, I actually, I actually have just. One question because I mean, I believe this crossed your mind. Didn't you ever try to a bit the script? Um, did you, didn't you ever think of like putting that word doc through a Claude or whatever and say, Hey, me the specs, the spec suite for this thing and work from there.
Ryan:Yes, I did. I did think about that. And I do occasionally have AI clutter, otherwise write some tests for me. Um, and, you know, maybe this is my own head trash. You know, I've been Jason's sweat, uh, and the, the rails, uh, his podcast, the TE and all his testing stuff time, that's what sort. You know, that way of thinking convinced me to put effort into building a test suite in the first place. And a lot of the other sort of developer improvements we've done over the years, he has a very specific way of thinking about testing, which is, you know, you're, you're writing a specification, um, which I don't always feel is how. The Rails ecosystem talks about testing and therefore when I put it into ai, I get back tests that I'm not super happy with. Right. Sometimes it's okay. I, I think it does a decent job at unit testing. I. My test script was all like, you know, system te like, you know, a browser level test, like using Capybara to click through and like, when a user does this, I expect this outcome and I found it's not as good at that. So for the time being and until I have more time to jump into it and you know, maybe train the AI a little bit more on my style of tests. I will use it for unit tests. I'll use it to test a class. Uh, you know, obviously that's a, a, a unit test. Uh, but I don't typically ask it to write the browser test or the feature spec. Um, I find it doesn't do a great job there. Um. And I think the other reason that that's been problematic, the only reason I chose feature specs is because unit testing our app is, uh, for the existing code is difficult right now because of some of the way I've written other people have written the code in the past, right? So I, it's again, a chicken and egg problem. I need to refactor it to make it better to, to make it more testable. That's a bigger problem because I don't have tests. for now it's like, all right, my plan. In general has been all right, I need smoke tests. That's what you know, that my doc is what I consider smoke tests. And you know, the one nice thing about to go through that doc for each of the rails upgrades we did is like, I realized what's important and what's not, right? So I probably cut that doc in half from the, the first time I ran through it. It's like, all right, if I test this, that actually covers a lot more than I thought. I don't need to do every permutation of this as I go through. So now I can, my goal is, all right, let's write these smoke tests as feature specs. Then I can have a little bit of comfort in going back and refactoring, you know, models or classes in order to make them more testable at the unit test level. And then I can maybe even back off some of the, future tests in the future, which are more expensive to run and make our test suite run, uh, you know, take a lot longer. This, this is the way I'm approaching it because there was no test thing when I started. Right. So like any additional test is a bonus. Right? And that's the way I'm thinking about it. So I gave you a little bit more than maybe you asked, but I thought people might be in the thought process.
Mateus:I was just, I was just curious if, if you did try it.'cause like I also think, um, at least
Ryan:I.
Mateus:I would not expect AI to, to be, some reason, I would not expect it to be very good at writing, like old test suits or whatever. Yeah. I don't know. I, I need to check out how Jason Sweat thinks about these things.
Brendan:He is got a whole book on it.
Ryan:there's a couple things. One is he organizes his test files very differently than sort of the rails way. So instead of having folders for like feature and model, he organizes it. You know, by the domain, right? So it's like, you know, I have a blog area and within that blog I might have unit tests and browser tests. and that actually makes total sense to me, right? Like I, I generally don't like to go against the rails way. But the workflow of being able to be like, all right, I wanna run a specific test to one whole file, to one whole, you know, domain of the app and then the entire app suite, um, is sort of almost impossible. And the sort of, you know, the quote unquote rails way of doing it. So because I organize my test suite like that, AI tends to trip up and I have to move files around, which I'm sure can be solved with. You know, some, some markdown files at some point when I get to it. The other thing is just like, it's it, and it is a struggle, honestly, for me, thought process wise. But you, you gotta think in terms of like, what are you, what is the behavior? That you're testing, not like, Hey, I'm testing this method. Right. And Jason does a much better job of explaining the nuance of that than I think I can articulate. So if you're interested, go read his stuff and listen to his podcast and, and consume knowledge and make up your own opinion. But that, I feel like this idea of testing the behavior you want is gonna be critical. As you continue to adopt ai, right, especially if you ever think, you know, you're in, if you're in the, the mindset that AI is only gonna get better and eventually it can actually be a competent coder. Um, you know, being able to write tests that describe behavior versus function, uh, for lack of a better term, I think would be. The only way that you could make that work. Um, but, you know, maybe that's a whole separate conversation. Um, and I know we, we gotta wrap up here, uh, in a couple minutes. So, uh, is there any place that you want, um, you want to promote? Uh, where can people find you or maybe you don't want to be found and that's totally fine too.
Mateus:Yeah. Um, I think.
Ryan:Or you can, you can pitch fast, ruby, or, or OmbuLabs too.
Mateus:Yeah, no, I mean obviously check out envelopes and fast, Ruby, please. Uh, we have, we do a ton of things Mo most people don't know this, but we do other things aside from upgrades. Um, so, you know, definitely check out our services page. myself, uh, well I do have a Twitter. I do have a LinkedIn. But they usually don't see too much movement. However, you can just search for my name. I'm Mateo. I guess we'll just leave that in the show notes. I can send you the links and you just put it there.
Ryan:Perfect.
Mateus:and I actually have a blog, which I've been trying to write too. Uh, and that one is easy. It's um, pair programming.blog
Ryan:Nice.
Mateus:my name means pear tree. So, you know, peer programming, why not?
Ryan:All right. Well, thank you for joining us. It's been a pleasure. And, you know, maybe we can do this again sometime and, and dive more into the testing, uh, piece that we, sort of left dangling there at the end.
Brendan:Yeah. Thanks for joining us.
Mateus:Alright, thanks for having me on. See you later.
People on this episode
Podcasts we love
Check out these other fine podcasts recommended by us, not an algorithm.
Code with Jason
Jason Swett
Standing in the Fire
Very Good Software
Remote Ruby
Chris Oliver, Andrew Mason, David Hill
IndieRails
Jess Brown & Jeremy Smith
Startups For the Rest of Us
Rob Walling
Vibing with AI Code
Brendan Buckingham