Part 1/2

In this video, the speaker begins by acknowledging the pleasant weather and transitions into discussing recent work at Snowpal, focusing on the implementation of API endpoints and UI tasks. They outline the challenge of managing numerous endpoints, both unique and common, and describe their approach to tackling this complexity, emphasizing productivity and strategic decision-making. The speaker explains their hybrid implementation strategy, prioritizing certain resource types while anticipating the need for refactoring. They highlight the importance of understanding the problem and making informed decisions throughout the development process. Overall, the transcript provides insights into the intricacies of API development and the considerations involved in managing a diverse range of endpoints effectively.

Summary

API Implementation:

• Detailing the delayed implementation of API endpoints despite completion of UI work.
• Explanation of the importance and complexity of the feature being worked on.
• Introduction to collaboration between UI and API teams.

Problem Statement:

• Describing the challenge of implementing numerous API endpoints.
• Differentiating between unique and common endpoints.
• Explaining the hierarchical structure and variations in endpoints.

Approach:

• Outlining the approach taken to address the problem.
• Mentioning the handoff process between UI and API teams.
• Brief mention of tools used for API development.

Implementation Strategy:

• Discussion on structuring the implementation of API endpoints.
• Exploring considerations such as resource vs. HTTP verb-based implementation.
• Emphasizing the importance of productivity and decision-making in implementation strategy.

Combination Strategy:

• Describing a hybrid approach to endpoint implementation.
• Example of focusing on one resource type before refactoring for others.
• Rationalizing the decision-making process behind the approach.

Refactoring Considerations:

• Anticipating the need for refactoring during implementation.
• Reflecting on the decision to prioritize certain endpoints.
• Illustrating the impact on addressing a significant portion of unique endpoints.

Podcast

Check out on Spotify.

Transcript

0:08

Hey there, hope you're doing well. It's been a while actually since I've done a video outside, I think. At least not when the weather has been close to 60°. It's absolutely beautiful today. Yes, I figured I'd just, rather than do the video from my office room, I'll just step outside, catch a bit of a walk, and just talk about something I've been working on recently.

0:33

So hopefully you can hear me alright and I've got my mic. But hopefully, it's not too bad. You might hear some background noise. And before I get started, if you haven't checked this out, go to snowpal.com, download the app from the App and Play Store, and just get yourself organized and enjoy all the features that we offer on our project management platform.

0:52

OK, without further ado, let me get to the content of this video. In this one, I just wanted to talk a little bit about the work I've been doing the last few days. And again, I've shared this before. Every week at Snowpal is actually very different from the week before or the weeks to come.

1:12

And even during the course of that week, we end up working on a variety of different things across stacks. So this particular week or the last few days, I've been spending time working on implementing API endpoints. The UI work for this really cool feature I'm pretty sure you're going to love.

1:29

This has been complete for a little bit actually. I intended to work on these endpoints a month, maybe two months ago, because we completed the UI part of the work for the most part, at least. But again, we had a lot of other things.

1:45

We've been working on a bunch of other things, so I couldn't get to working on these actual API endpoints. So now I figured I'll circle back because we want to get this out the door hopefully in the next maybe four to six weeks. It's a pretty big feature and it comes with a lot of other enhancements as well.

2:04

Anyway, so to get to the actual problem that I want to talk about here, the UI work was done. And here's an example of one of the many ways you can actually collaborate between the UI and the API. Teams can actually collaborate, right?

2:20

This VR cell's use a variety of examples and scenarios. It's just one of them and hopefully, it resonates with you and it helps you. So in this example, it's a true real-life example, right? The UI work has been completed, right? Pretty much, other than the integration part, the UI is connecting to API like blueprints, stub dot APIs, and responses.

2:44

So that's how the UI team worked on it. Now I'm getting to, after they do that, our process is that they would create a specification. We have a large number of endpoints in production already. I've mentioned that before. This adds about 75 endpoints I think to our existing list and not all 75 of them are actually unique.

3:09

I would say about 30% of them are unique. And what I mean by that is 30 of those endpoints almost have nothing to do with each other in terms of implementation. But the other 70%, they have variations of the same type of endpoints, right?

3:27

So because we have keys, blocks, paths, and block pods, we have three levels of hierarchy and in some cases two, but we support up to three levels. If you haven't checked this out, just make sure you create parts under blocks and you'll see what I mean. So when we implement a feature, sometimes it's specific to a particular type of resource that it's a key block or a pod, but other times it is common across all these resources.

3:53

In this case it is. So there are like 75 endpoints with 30% of them, give or take, that are unique. So how do you go about solving this problem? Right. So the approach I've taken, we've taken actually here in this particular scenario, and if this changes depending on the feature, depending on our milestones, deliverables, yada, yada.

4:14

But for now, so the UI teams completed their work pretty much they create this specification, update the spec to add these endpoints. They tell the API team and in this case they are RESTful endpoints and in other cases could be graph endpoints, right?

4:31

So they tell us what we need to do, what those endpoints are, what the actual path is, what query string params or request body you might need to pass for POST or PATCH request, whether it's PATCH or PUT and what are the variations of the deletes?

4:47

We support the whole nine yards essentially, right? So they publish those endpoints via the specification and that's exactly the same. And those are the same endpoints that they stub out on API, right? That's what they're using. And then the specification also includes.

5:04

This is like, you know, nothing different, right? It's like publishing a Postman workspace, but I'm not a huge fan of Postman, so I use Insomnia, which works very well for Graph and REST APIs and I've been using them since. They actually probably went live, I think.

5:19

Or it's been like three years. I've not done any. My all of my Graph work has been using Insomnia and I actually fell in love with the two the client, because I like Postman. It doesn't open like a million gazillion tabs. It's a whole lot easier to use and I personally prefer it.

5:35

It's just a personal preference anyway, so we, it's got its pros, the cons as well because they don't support open API so we have to go through some hoops to do the conversions. Anyways that's besides the point, so the UI team publishes the spec and then they give it to the API team.

5:53

Now that's the handoff right and any documentation that we include as part of the specification. But you know, given that we've done implementation a large number of endpoints we kind of know what to expect, right? It's just the endpoints are different, the expectations are different but the documentation we don't spend too much time on it.

6:11

It's in the spec or we discuss it over a phone call and we are good to go and we kind of carry it over to a GitHub issue and crack it on a GitHub project. So now that they've given the endpoints, what

did I do the last couple of days, right? So the first thing is we have a structure in those endpoints.

6:30

So those 75 endpoints are broken down into different directories, sub-directories. It does what to do and you can almost rely on it for the sequence of implementation but not entirely. So here is why, right? Let's say I'm implementing a POST for a key for that particular endpoint.

6:47

For that feature, I would have to do the same thing for a block, a pod, and a block pod. But then there are other endpoints, right? The other POST endpoints, other PATCH, PUT, and DELETE endpoints. Now you could either do it resource by resource, or you could do it by the HTTP verb.

7:07

That's your choice, right? And you figure out what might make you productive in that scenario. And these are very important because these are decisions that sometimes API developers don't give enough thought to. So what happens is you may just start implementing it and then they start reporting statuses.

7:26

But the question is, are they taking the most optimal approach from a productivity standpoint? And it depends, right? There's no single right answer whether you want to implement all POST requests first before you go to PATCH, or whether you want to do it by the resource.

7:41

It's entirely up to you. That particular problem, the complexity of the problem, your deliverables, and it's a combination of factors, right? Unfortunately, it's not like it's not a syllable bullet or there's not a single straightforward answer, but regardless, you have to have a hunch as to how you want to go about it. In this case, I actually did a combination of the two.

8:00

I first worked on the key endpoints, right? So it was across HTTP verbs for that resource. But after I did that, I knew for certain that when you go to implement the same type of endpoints for blocks where there are similarities, you're going to have to do a bit of refactoring because the code is going to have to be similar.

8:20

Because initially I created all the files in the scaffold and the skeleton of everything that relates to this feature, I typically stubbed all the files, right? So in this case, I had to create like what, 35-40 different files, models, the routes, the services, you know, everything that you typically need to implement an API service.

8:43

But when I did that, I knew without any doubt that when I would go to implementing blocks, I would most certainly have to do the refactor. So I asked myself, not literally, but very implicitly. But I wanted to be able to do that, because if I did not want to do that, then I'd have to implement all the POST endpoints at the same time for all the resources for that specific business problem.

9:05

In that case, you would write it sort of correctly the first time and not have to refactor. But in this case, I made the conscious decision to actually get the key part done, the key part, like not, no pun intended, but the key part of the key resource in our product done.

9:21

Because I figured that if I implemented keys in their entirety, that would take care of about 60-70% of the uniqueness of those unique endpoints which were 30% to begin with. So just to get back to the math because this is very important for you following this closely, if you have like 100 endpoints or in this case it was 75 endpoints, actually about, I said 30%, right, that's like about 22-25, let's say 25 well.

Part 2/2

The speaker outlines a comprehensive approach to time management and problem-solving, emphasizing the prioritization of complex issues. It begins with an introspective discussion on time management challenges, followed by a progress report detailing the strategic implementation of key endpoints. The narrative delves into the intricacies of refactoring and implementation challenges, highlighting adaptability and forward planning. Collaboration with the UI team is emphasized, underscoring the importance of clear communication and integration testing. The conclusion reflects on the journey, acknowledging growth and offering well wishes to the audience.

Summary

Introduction:

• Introduces the concept of implementing key endpoints to address complexity.
• Emphasizes the importance of addressing complex issues first.

Progress Report and Strategy:

• Reports progress on implementing endpoints, focusing on key endpoints first.
• Explains the significance of solving unique endpoints first.
• Discusses the approach to solving the problem and the time required for implementation.
• Reflects on the importance of pre-planning and thinking through problems before coding.
• Mentions future tasks and potential time savings through optimization.

Refactoring and Implementation Challenges:

• Details the refactor process and optimization techniques used during implementation.
• Discusses the flexibility in approach depending on the problem's complexity and timeline.
• Mentions remaining endpoints and plans for integration testing.

Collaboration with UI Team and Next Steps:

• Explains collaboration with the UI team in defining specifications.
• Mentions upcoming tasks for the UI team and integration testing.
• Acknowledges potential benefits of visual aids for understanding complex topics.

Podcast

Check out on Spotify.

Transcript

0:06

Right. That's like about 2225, let's say 25. Well, yeah, 25 would be 33%, but let's say close to 25 endpoints that are unique. Now, if I implemented the key resource, all the endpoints related to keys in their entirety, then it's actually going to be like close to 10 minutes.

0:23

I'm going to, well, it's too late. It's gone past 10 minutes. So I'm going to have to split this and publish this on LinkedIn. Sorry, I didn't keep an eye on the time here. Anyway, so back to what I was trying to say.

0:40

I lost my train of thought there because I was conscious of the time here. I was just saying that the 30% would be like 25 endpoints. But if I implemented all the key endpoints, all the endpoints for the key resource, in other words, that would take care of 70% of the uniqueness.

0:59

In other words, I would only be left with 30% of uniqueness remaining. Hopefully, that makes sense to you because it's important, right? You want to deal with the most complex problems first because the chain is only as strong as the weakest link, as they say, right? So there's no point in implementing the same type of endpoints for all the resources, in my mind, at least in this scenario, because they haven't told me if there's a problem, whether there's an architectural challenge or if there's a design issue.

1:25

But by implementing the unique set of endpoints first, not being agnostic of the research, I mean, just taking all particular resource, you try to handle those difficult issues first. That's what I did the last couple of days, and now I'm pretty much done with the key endpoints. In terms of the count, I'm only like maybe 15 endpoints in, give or take, right?

1:45

So if I were to report this as a status to myself, it would almost seem like you're 20% done. But that's not true. This is where ETS goes out the table, and I've discussed that in a different video because that's not the way you interpret this, right? You're sure there are 75 endpoints, you're done with 15 of them. You do the simple math, it's 20%.

2:05

But that's not it. Because these 15 endpoints are unique endpoints and they are the first ones you solve, and it's always more difficult to solve it the first time than the second time because you're just following a design pattern. At least that's an idea. So when I go to implement the same set of endpoints for the other resources, it's not going to take me as much time as it took for this one, right?

2:25

For this one, I want to say I did a lot of refactor before I started the work, so I can't include them. But let's say it took me, I want to say about maybe 6 to 8 hours, give or take, for just that particular work, right?

2:40

And that's all, not. And you don't get to do it that quickly because unless you think about this problem upfront before you get to your desk to code. And I also discussed that in a different podcast. It's rarely, not even rarely. It's I can never.

2:56

I cannot think of a scenario where I've actually gone to my desk and sat in front of a machine without kind of having a good idea as to what files I would need to create, how many of them there might need to be, what the logic should be, right? So I've kind of thought through all of this without being in front of the machine.

3:13

So when you actually go to the machine, you start typing, literally, you start typing, right? And that's why. That's one reason why I think sometimes we're able to get things done a little bit faster and you should. If you haven't seen that podcast, I encourage you to check it out. I actually, I've kind of explained how I do this thinking while I'm walking or working out or whatever else it is that I might be doing.

3:34

Anyways, back to the problem, so 15 endpoints done. I have another 2025% to go for in terms of the uniqueness because that's brought to the table by blocks, pods, and block pods and the hierarchy in itself. So I cannot solve that uniqueness by virtue of the endpoints related to the key resources, right?

3:51

That's fine. So I would do that when I go to blocks, that's the next set of things. Now let's get deeper right into this discussion with the endpoints that I implemented right now. I mentioned earlier that I expected for that to have to have some refactor and it sure did because I started doing a couple of block endpoints and I quickly realized I can actually generate size a lot of the code.

4:16

And this particular API code I'm writing currently is in Ruby, so if you're familiar with Ruby, you know it supports. It's very rich when it comes to meta programming so you can actually remove a lot of boilerplate stuff and redundancy and keep code as DRY as possible.

4:32

So I've I've I did a little bit of refactor so the key implementation and the block implementation could actually share the same you know the same services or the same base services and base models and whatnot.

4:49

And I think that you that time could be potentially saved if you had taken a if I had taken a slightly different approach but it didn't take all that much. It probably took me like 30 to 45 minutes to do the refactor and that's what I expected. So I think I was I intentionally picked this this methodology for this implementation but that's that doesn't mean I do the same thing.

5:09

I do things the same way every single time. It literally depends on the problem and our timelines and the complexity of what we're trying to solve. So that's. So let's say just to wrap this up, I have another like 60 some endpoints to go and once we're done with that, obviously I would push the code to our lower environment and then we start doing the integration testing.

5:32

And it's before we even we do that, we don't necessarily absorb everything that the UI team gives us in terms of the specific, the specification. A lot of times we publish the spec, that's the majority of the cases. But in this case, I work closely with the UI team to define the spec right.

5:50

So it certainly has the inputs from the API side, but the choice of attribute names and stuff in this case the UI team picked it, right. So what that means is there are certainly some changes. I noticed I had to make those changes, some minor adjustments to the endpoints, the pad, the URLs and what not.

6:07

So the UI team will have to make those adjustments. It's probably half a day's

worth of work, maybe a day tops. Once they do that, we would push that to the lower environment and then they would do the integration testing and see where if things, where things fail and make those adjustments right. But here I've walked you through.

6:24

Maybe I can draw it out the next time, but sometimes you're outside and you can have the best of both worlds. It's probably much easier to see it visually. I know people have given me that feedback, but I mean, I'll do it when I can. Yeah, that's that's a true that's a real-life approach to implementing APIs and a large number of endpoints for a system that is in production and we're adding a brand new feature.

6:47

This is not a bug fix, it's not an enhancement, it's actually a new feature altogether. So hopefully, hopefully this helps. I know I try to sometimes. I pack a lot of content into these videos, and I've had that feedback from more than one person and I have the gin.

7:03

I honestly try to not say too many things, but sometimes you have to click record. You just forget, you forget all the learnings and feedback and you just go back to it's like go back to the original shape or something. But anyways, it's, you know, live and learn as they say. Hopefully wherever you are.