Microservices are one of the most popular modern architectural approaches, but they are much more complicated to do well than most organisations think. So what is Microservices Architecture, what is it for, what are Microservices and why are they a lot more complex than they look on the surface?
In this episode Dave Farley will explore the costs, and benefits, of microservices listing the key features that distinguish this approach from others and answers questions like "When are they a good choice” and "when are they not?”. One of the biggest problems with this useful software development approach is that many, maybe most, organisations that try to build a microservices system, aren’t building microservices at all! It is more complicated than it looks!
-------------------------------------------------------------------------------------
📚 BOOKS:
📖 Dave’s NEW BOOK "Modern Software Engineering" is now available on
Amazon ➡️ amzn.to/3DwdwT3
In this book, Dave brings together his ideas and proven techniques to describe a durable, coherent and foundational approach to effective software development, for programmers, managers and technical leads, at all levels of experience.
📖 "Continuous Delivery Pipelines" by Dave Farley
paperback ➡️ amzn.to/3gIULlA
ebook version ➡️ leanpub.com/cd-pipelines
📖 The original award-winning "Continuous Delivery" book by Dave Farley and Jez Humble
➡️ amzn.to/2WxRYmx
_____________________________________________________
Also from Dave:
🎓 CD TRAINING COURSES
If you want to learn Continuous Delivery and DevOps skills, check out Dave Farley's courses
➡️ bit.ly/DFTraining
Get a FREE guide on "How to Get Started with Microservices" by Dave Farley, when you sign up to our mail list: ➡️ www.subscribepage.com/microse...
The best way to keep up to date with the latest discussions, free "How To..." guides, events and online courses.
---------------------------------------------------------------------------------------
Other Useful Books on this topic:
Building Microservices: Designing Fine-Grained Systems, by Sam Newman ➡️ amzn.to/31PyXOS
Monolith to Microservices: Evolutionary Patterns to Transform Your Monolith, by Sam Newman ➡️ amzn.to/35IB8EO
(Please note, if you buy a book from these affiliate links I get a small fee, without increasing the cost to you)
Focus on the right parts of the problem when you are creating a new microservices system. Here's my FREE 'how-to-guide-book' offering advice on the Microservices basics to help you get started ➡www.subscribepage.com/microservices-guide
Expectations: Rant about microservices Reality: Misuses of microservices and advices how to avoid mistakes 10/10 quality content!
Yes i was expecting the same but found gold 👌
Reality: Clarity on the essence of MS (mistakes and misuses come from that lack of clarity). Still 10/10
The same thing happened to me. 100% quality content indeed!
I agree. The title is misleading. It is very good content!
Copy that
I’m a 30+ years Software Engineer, with clearly a similar path. This is extremely well articulated and should be on the ‘reading’ list of all modern software developers who want to understand concepts from a big-picture level. I don’t often comment on KZhead content but this is worth expressing thanks for a good, down to Earth presentation.
Every developer out there should watch this video. Orchestration and monitoring tools vendors made a lot of damage by trying to get everybody adopting microservices so they could sell their shiny tools. Then developers found themselves producing distributed entangled spaghetti aberrations just because nobody taught them what Dave excellently does here.
Thanks😎
Developers are usually very aware of the problems and find themselves producing distributed spaghetti aberrations because they are not the ones who makes such decisions. Typical victim blaming.
@@linuxgaminginfullhd60fps10 It seems to me that @Fran Dayz was saying tools vendors luring developers to purchase their product and then not providing proper support was the issue he had experience with. But you are saying that decision makers forcing those tools onto developers is the issue you are facing? I'm sorry, I do not understand how those situations are the same in a way that is offensive. How does any of that change the point made that Dave's explanation here can help avoid aberration creation in the first place?
As always: Don’t solve Problems you don’t have 😀.
Thanks a lot for your videos. Besides the content I really appreciate the way you speak. Somehow it feels that are using the words of the English language, optimized for simplicity and information throughput.
Wow! With almost 25 years of professional software development under my belt, this was the first video/document on Microservices, which made sense to me. Everything else, and that includes most of the big software conferences, was just a variation of marketing bullshit. It is really fascinating to see that it always comes back to how we deal with "complexity". And that is not technical stuff, but the organizational and domain side of things. Thanks a ton!
Thanks, I am pleased that you liked it
YES! Microsservices is a solution to a problem that the majority of companies never had and probably never will.
@@rafaelrosa3841 it absolutly is, otherwise why would every big tech company use it ?
agree
Would have been even better if presented by Elon Musk
I have been interviewing with a lot of companies lately that want candidates with micro services experience. Unfortunately most of the companies don’t really understand micro services. They think it’s the NEW way to do development. What they don’t realize is that micro-service’s is just one of many ways to implement your project. Great video!
I'm a 25 year Master Journeyman plumber...but somehow have become fascinated by microservices over the last few weeks. I absolutely never plan on coding a single thing in my entire life...but really enjoyed this video. Gives me a little perspective when I update some software and wonder how they introduced so many bugs with a simple small update...
It's remarkable how similar are system architecture and plumbing. Or, at least how I imagine plumbing works. I've come to believe that civil engineering falls in that group too. Basically anything that applies patterns and practices to the management of flow and complexity.
This channel is amazing, thank you sir! I am arguably a seasoned engineer at this point but listening to you keeps me motivated to better myself constantly. I think will watch every single video of yours. Keep up the good work!
The way he explains things is amazing
Great to hear! One of my enduring principles is that we all need to keep learning! and to use software development techniques that help us learn. Thank you for the feedback.
I clicked because I saw Captain Dissillion's logo.
One problem that I’ve experienced with microservices is that it is really hard to keep everyone up on what are the boundaries of each of you services… sometimes a lot of requirements come from the managers and they want it do be done in service A when it should be done in service B and not all employees know all of the microservices that well to be able to set those boundaries… I’ve experienced features being done in a service and then it came up that the same feature was already being done in another service that is called way up in the architecture, way after it was already shipped to production. I’ve thought about it if that’s a problem of not having so many people understanding that well the architecture or if that’s a leadership or management issue… I feel like there’s a small inner system that developers are capable of understanding, it’s almost impossible to master the design/domain of so many of the microservices. I haven’t seen anyone talking about this, I’m always wondering how companies like Netflix deal with that, like, how many other mircroservices do your engineers need to grasp besides the one/ones they are really contributing code to?
Really great stuff! Thanks so much! Working as a management consultant I see my projects 'moving closer' to the field of technology every year. Thus, usually I can't deliver valuable results for my clients without understanding related technological practices and principles. Your videos are amazing in the way you are able to explain those technological issues! 10/10
Your videos are extremely valuable to me especially as I'm going down the software/systems developer route. Thank you for sharing knowledge.
My development is mostly a private affair. I write code that I use, web interfaces that I am the only customer for. This content is still helpful, even for my hobby. This video got me thinking of problems that should be decoupled that aren't, and conversely convinced me not to decouple things just to make them "micro." There is no shame in coupling my front end with my back end code, for example in my latest node.js project.
Starting monolithic and transitioning to a microservice architecture seems to be a common route when the team GROWS- Because in an ideal world, each microservice deserves it's own team.
I can't with how valuable this content is
Thank you! I'm glad you find it useful
It is best explanation found so far. Thank you very much
You can't what?
I've watched a number of your videos now and they are brilliant. Straight talking and oozing knowledge and wisdom. From now on, part of my decision process is going to be 'what would Dave Farley do'
Thank you 😁😎
The explanation of the bounded context is better than any other I've heard.
The only software engineering processes and practices related content that makes 100% sense. I've been a part of multiple discussions around such topics like software architectures, design patterns, agile development, behavior/test driven development etc...none of them make as much as sense as much as your content does. Dave, please continue posting such videos. I want you to know that your experience, talks, and discussions make lives easier for many software engineers worldwide! 🙂
My key takeaway here is this: the API (as in the names of the calls, the expected arguments, the expected results, etc) should be the first thing to be designed and agreed upon when the analysis phase is over. On top of that it must be cast in stone, immutable and stable during a development cycle. Finally it must be the first thing that will be mocked during implementation so the service consumers have something to work upon. The mentality to adhere to all of this in an agile environment where there are constant releases every e.g. month or so, this is the real fight, this is the real issue, the fact that people must put down some communication constraints, draw a box and remain in there, no "well, you see, the customer also decided they want this and this and this so we are mutating the API definition". A mini lecture on how to keep people in check (both on customer side and on the PM/TechLeads side), now, that would be a treasure!
I think having any design that is inmutable and cannot adapt to customer demands should be a definite no-go for any serious software project. Instead, having stable APIs means that once you create an API, you have to keep it running. To add new features, you introduce new APIs without breaking the old API. Some of the old APIs may be implemented as wrappers for the new API but the consumers of the microservice API don't need to mind about that.
@@MikkoRantalainen Introducing a new API for every change isn't practical either. It leads to growing legacy code base, as it's problematic to find out when it is safe to delete some old API version. That's why big companies end up with monorepos, as in a monorepo there's more control over the consumer's code, as far as the API does not escape the monorepo.
@@user-dt9xb7sn2q I'd argue that if you continously modify the API in a way that requires consumers to be modified, too, you don't actually have an API at all. You just have fully custom protocol instead.
As a relatively new developer (2+ years) I really enjoy your video. Helps me to think at a scale larger than the code immediately in front of me. Thank you for this!
It's a pleasure, and thank you. As a new dev, have you checked out my video on "Career advice"? kzhead.info/sun/m86CnLiZeH-plYE/bejne.html
The idea of "bounded context" is so important. Whenever i discuss architecture with people i talk about my idea of a "module boundary" but i always struggled to explain what i mean. This description of a bounded context is exactly what I was looking for.
What a fantastic experience is to hear explaining theses concepts from someone who already implemented microservices in many contexts!!!!
You're channel is a gold mine for CS students, thank you so much for sharing your knowledge!
I really appreciate all of the work you do to provide clarity and context to the field of technology! As of recently, I landed my first full-time job as a software developer, so this is providing excellent information as to how I can pursue my career with a strong foundation. Due to being early on amidst my CIS education, several of these topic areas are new to me which provides its own set of difficulties regarding comprehension, yet I always find a plethora of solid resources from these videos to help elaborate on any confusion. So, again, thank you for setting me in the right direction.
Thankyou, and congratulations on your new job. I have a new video coming out in a couple of weeks that is intended to offer some advice for new developers, so I hope that you enjoy it.
@@ContinuousDelivery Looking forward to it!
Working on something and just found this great video. Thanks for it. I was already using his advice on 12:41 and will continue to keep it that way until the need grows. If this is not late, maybe advising on how to handle cases on communication between modular designed services will be awesome.
Solid presentation. Everything is spot on! Congratulations on tackling this misunderstood topic very elegantly.
Your experience shed light on exactly what I felt was off about people/companies claiming to be doing microservices but most definitely are not. This is an enormous problem for semantics and execution. Not everyone is on the same page. I really think of big monolith companies that want to decompose their design into "microservices", but in the end they will spend a ton of money for little to no benefit (ROI). So much time is often wasted chasing after 'trends' before thinking things through...
Yes I see a lot of orgs investing a lot for little in return too.
I don't think their chasing trends, their chasing ways to bring "complexity" somehow under control (probably because they did not pay much attention to that previously at an organizational level, which resulted in a lot legacy code/systems) without understanding all the implications of their decisions and approaches, and as Mr. Farley said, there are many other ways to do that besides microservices. But the change is not because it's trendy, it is because the people involved feel the pain in developing and maintaining said legacy system. The company big wigs don't care that much about fancy code or architecture they care about money and they were ill advised that this new shiny architecture will save the system from crumbling from within (and also solve all world problems from global warming to the inevitable demise of human kind), and in the process they sometimes accelerate the decay of the system by over-engineering stuff (that also get polluted by the old way of doing things).
@@nerrierr I think you missed my point entirely by focusing on a single word. Microservices DO NOT attempt to bring complexity under control, it introduces more of it. The core purpose is for scalability and team environments to work on separate code bases in isolation but will work with one another when connected. Monolith and Microservice design are essentially antithetical to one another when done properly. But the point I was making was that companies using monolith design are decomposing their code into microservices for the wrong reasons and doing so in a hybrid way which gives little to no benefit for all the increased difficulty. Doing so is following a trend: see technology adoption lifecycle.
@@destroyonload3444 fair enough, but to be honest that still is directly related to the complexity and requirement of the project. You don't scale and parallelize if you don't need to (ex: for a simple console application that just prints a simple message you will not do it in a microservices architecture no matter how misguided you are). As for the fact that it adds more complexity it is like saying that abstraction/encapsulation just adds more classes/methods without a benefit (instead of helping you in the grand scheme of things, nothing is free in life and in this case you pay some extra code "locally" to save your a*s some place else, and you do it because of the complexity of today's systems). In the end I think we agree that you need to pick the right tools for the job and for the right reasons, the difference is why we like/dislike microservices (in that sense it's probably relevant to what we are comparing it to). And that is perfectly fine, no team or project is the same in every aspect and people have different opinions based on their previous experiences (there's a reason why giant companies recommend it, and in fact only they are, but most companies aren't Netflix nor is their product of the same scope/context) (well one of the reasons is obliviously practical and the other is probably more marketing focused but that's another topic, I guess we all have to earn our bread somehow :) )
@@nerrierr right. You still have abstraction in microservices, but the real complexity comes with multiple middlewares and dealing with events. Statistically, I have no idea what the most used implementation is, but the most I hear about is the use of an event server and eventually consistent design. I think we're more or less on the same page. BTW, I watch this channel often because his methodology seems to make the most practical sense in implementation, and he's been through several major eras of programming milestones. I find it funny that people are still trying to reinvent what he already had working in the 90's! I watch a lot of Computerphile, too. The down-to-earth "old timers" have my utmost respect.
I almost never comment. But I must. This is a must watch by ANYONE involved in solutions development in a modern organization. 10 out of 10. This should even be watched by the non-technical stakeholders. Well done.
Very well explained, as an absolute beginner in software development, I find your videos easy to understand and thought provoking! Look forward to many more videos from you :)
Very good video that captured a lot of the discussions I've had surrounding misused microservice terminology very well. Just because something isn't a pure microservice doesn't mean it's bad, but it does mean you should find a different term for it or people will get the wrong idea.
Thanks for the video. The idea of a ‘distributed monolith’ is what i find intriguing and I think it is what we are building where I work although we ‘think’ and ‘say’ that we are building micro services. The benefits however are still those of a distributed system: scalability, resilience etc, and maybe that is all we really wanted. :)
I feel that the font used for the presentation points is the same font used in the Star Wars opening text crawls, and it is immensely appealing.
A long time ago, in a galaxy far, far away.
I deal with large deployments and this contents is spot on 100% accurate.
This is fantastic stuff, as someone who's changing career trajectory having this high level overview is very helpful. Thank you
Hey Dave, great video. I've often heard the "2 week rule of thumb" for rebuilding a microservice, but one thing that was never clear, was whether that was an individual doing it alone, or a team? Be great to understand that. Thanks for all the videos and contributions you've made!
Philip, I think the timeframe is 3-10 business days for a small team (3 people). I’m just guessing on that but this is just meant as a rule of thumb. The idea is that you dont want more than say 150-200 hrs of work to go into a microservice. After that you are really talking about medium-services 😂. Smaller is better here. Another rule of thumb I have heard is that you want no more than 6-10 methods/endpoints.
I think a couple other points that destroy microservice implementations are a tendency to break services at business unit boundaries and a tendency to delineate horizontally almost exclusively. I think you touched on the first a bit in the video indirectly when talking about how systems mirror the communication channels of the organization, but maybe not as explicitly. On the second point, organizations don't tend to build services that, as you said, do things like storage, but rather each business domain service implements it's own (and often multiple) storage subsystems.
I am glad that introduced microservices back to 2000. And after 21 years finally I found your video. Thank you.
Thank you very much for the explanation ! very valuable and straight to the point !
Great content Dave and well laid out. Your trajectory of your career is basically the same as mine, I could steal that slide to explain my journey. One difference is in the early to mid nineties I spent time on shrink wrapped software and then operating systems. The one caveat I would say to your whole thesis is just that there is no silver bullet architectural pattern. Micro services are great for the reasons expressed but ONLY if it solves the problem domain in an elegant and sustainable way. Every pattern has its use and I’ve seen teams try to use micro services to “evolve” legacy processes like batch jobs to “modernize”. Guess what? The Batch job, even in this century is still a legit pattern in some instances. There is no one size fits all and in my experience blending or cherry picking the past of different patterns is often the right thing to do. Religious adherence to a pattern can make your system more complex if you first don’t explore multiple options and aren’t predisposed to have to design with the currently fashionable pattern or technology. My favorite example of this faux pas is XML. I’ve had to fix many systems because everyone assumed XML as a data interchange was a requirement for a good system. It only slowed things down and caused tighter coupling with dtds, parsing code, and made interchange with non xml systems a nightmare. Nobody likes to write XSLT let alone debug it. New subscriber here.
Great explanation of what it is and what it is not a microservice, would be nice to add or to discuss the different approaches to actually make good decitions in the design of the bounded contexts and each microservice in more depth.
Thanks for this Dave. Yet another quality, informative piece of content. I've only recently discovered your channel, and I'm astounded as to how much we share in terms of engineering philosophy (maybe it's our age :D). I've started imposing a lot of your thoughts and ideas on my team recently, and they've (for the most part) been received well. The new year is going to continue that trend.
Great video! One issue that can be tough to resist with these things is when scope creep by changing needs gets forced onto the developers and breaks the autonomy aspect. Would love to hear your thoughts on how to address scope creep when trying to keep your projects clean and modular.
I think that you can't fix things forever, over time, the ideal division of responsibilities in code, and in teams, will morph. A thoughtful organisation will take this into account and re-assess team structure, and software architecture over time. These are deeply related ideas, and I have a video that gives one take on the org challenges here: "How to build big software with small teams" kzhead.info/sun/ltSPfN6boXV4YKs/bejne.html At the more practical, tech level, I think that as you discover new features, you decide if this fits within the current scope of your services or modules or if you need to add new ones. If you add new services, then at some point you need to decide if you should spawn-off a new team to look after the new services.
Man, this video has so much more quality content crammed on it's 17 min than a lot of books and courses.
I Love your format - slides make your content easy to follow and review, while you’re delivering in earnest alongside
What a brilliant video! The decoupling between services brings so many questions to me (that really likes modularized monoliths and haven’t worked a lot with MS’s). How to handle relations in between services, if at all? Are people storing IDs and just hope for the best? How to tie the pieces together? Via API GW? or independent calls to each service? Wouldn’t that be considered coupling since they’re no longer independently deployable? Would it not only be really decoupled by letting each service being complete (with UI and everything)? So many questions!
Thank you. Yes it does raise a lot of questions. I think that one of the secrets is to think very clearly of the "protocol" of information exchange, being separate from the service implementation.
Solid video. My compliments to you for simplifying a misunderstood and misused term amongst sales and marketing people .
Thanks Dave. As you mention, there is very much more to cover on this subject, but from a QA point of view the subject of contract testing needs to be raised early on in any discussion about microservices. Also there are some severe gotchas with the approach too. As with many things we need to do our upfront research or risk project failure! Sam Newman's books contain a wealth of info.
Yes I am thinking of doing more stuff on design and architecture on the channel. I don't think of it as "doing up front research" as much as understanding what the things that we do are, and how they work, we often we seem to jump on ideas based on sound-bytes and fashion rather than understanding. I link to Sam's book in the description and meant to reference it in the video, but didn't want to appear to put words into his mouth. Sam's book is a very good book on the topic.
Great video! What about duplicated code?
Great content and explanation of these topics. Thanks.
Really appreciated this video, should be a must watch for on-boarding developers in an environment with microservices implementations
Excellent video This reminds me a bit of agile, people parroting a new buzz word but failing to implement it spectacularly
As a person, who made microservices before people started calling it that way I am happy for this video. Separately deployable is something that I see so many people out there do not grasp. What I do not grasp however is sometimes how heavyweight the infrastructure is around real microservices! In my team I did not use any kind of infrastructure just plain REST and pretty much plain javascript (not even js frameworks) to implement these ideas and yet there was a clean way to achieve copy-paste kind of integration even on the frontend side - and by that I literally mean the code was just copied into its place and was magically working. Never saw a lightweight solution like that later, but only heavy solutions where they used some tool to "help" create microservices, but only then I felt the cost of it - not when we did the stuff manually without framework. I think some movement like "lightweight micro services" should arise. Maybe it is already arisen, but not what I saw. The idea is that if you cannot set up your microservice infrastructure in a week with everyone on the team understanding all the details - then you are doing it wrong. This is similar to what you say about the microservices themselves and their sizes, but here I am talking about the support infrastructure that make the deployment and everything work - both backends and frontends of the microservices and the ability to publish them even when working together! I hope such a thing exists, but as I said I only saw two things: we doing without any specific infrastructure OR some teams using heavy infrastructure that incurs more cost that simple projects can afford. This is a big deal because original company where we worked like that was less than 50 person and team around projects where we employed this was less than 6 person and still we did not feel at all "too big work" to develop in this way. Any yes again: it was completely shippable separately - even the frontend of these services were separately shipped and built up a complex web app. Anyways: I think everyone who ever use the word by their mouth in any circumstances should watch this video. Would help a lot if people would all do so...
That bell sound is so loud and ridiculous, and it always makes me laugh when I hear it. I could just listen to these videos on a loop all day and never get bored.
Thanks, sorry about the bell, I am making it quieter in future videos 😉
So hard to setup contract testing with MSA, this video highlit (to me) why I've been struggling lol, thanks
This channel deserves more subs and views ❤️❤️ Finally: A professional software content for everyone .... Greetings from Colombia.
Thank you very much!
Traigan el cafe berracos, que sea del weno.
Very good summary, thanks for sharing.
Starting my DevOps / DevSecOps journey.....this is helpful thank you.
Glad it was helpful!
Awesome insights. Glad I found your channel!
Nice and great video, all to the key points. Thanks Sir.
Hey Dave just discovered your channel and I love it! thank so much for creating this content. It is amazing !
Glad you enjoy it!
thank you. in terms of things I'd like to hear about - you mentioned how software design ends up modelling the organisations communications - I have encountered strong coupling between org culture and development culture - especially as a road-block to change. It's bleedin obvious to me that say moving from monolithic software to distributed software takes cultural change within your development team (eg. moving from test/deploy to CI/CD, or agile, or DevOps) - and communication/cooperation/coordination between teams needs to increase significantly... I'd love to hear about the sorts of approaches that make this change possible, success and failures, and so on - not so much a tech question I know but damn if it isn't the core of the problem...
Great explanation about Microservices. Thanks.
Clear many confusion, Thanks a lot.
Very clear presentation, thank you!
Very well described. Thank you!
Although I am a software developer with 22 years of experience I am new to microservices and this video really helped to improve my understanding, so thank you very much. My immediate conclusion, taking Melvin Conway's wisdom into consideration, is that an organization that wants to adapt to a microservices architecture should be willing to reorganize accordingly. If the management of the organization is not aware of this, the campaign of migrating towards microservices will fail (at least initially). So business consultancy (of good quality) should be involved to make this a success. What I have often seen, however, is that new paradigms or technologies are presented to managers as a panacea of all their current problems without any big effort on their part (just buy our product or pay our consultancy fee). Do you have any thoughts on that? How can a senior software developer advice or influence the management of an organization to do the right thing? (either endeavour a reorganization or recognize that microservices are not for them, yet)
Don't work for pointy haired bosses.
Totally agree: The most important part in using micro service architecture is organizational. Specially in context of independent agile teams these days. 😅
The biggest problem I see with microservices is how they deal with the underlying data layer. The problem with data is that as your software gets bigger and bigger - so are the relationships between the various data entities. Since each microservice is deployed and versioned separately - this means that you will not be able to utilize these data relationships effectively as each microservice needs to has its own small "data island" that is separate from all other data parts. This usually reduces your DB queries to the most primitive single table SELECTs - which means you will not be able to enjoy all the progress done with relational databases in the last 40 years or so.. this means no advanced JOINs, statistics, stored procedure etc. Unless I'm missing something here - in which case I'll be happy to see how this problem is being resolved in big microservice installations..
Part of the definition of "Microservice" is that they DON"T SHARE DATA STORES with other services. So this shouldn't be a problem. Microservices are responsible for their own storage needs. They only share information through their public APIs, whatever form those take. DBs, of any form, are specifically ruled out in all of the definitions that I have ever seen, as a means of data-sharing. See Martin Fowler & James Lewis' description, in which they describe this as "Decentralised Data Management" martinfowler.com/articles/microservices.html
@@ContinuousDelivery from my experience - using compensating actions and avoiding database transactions can become extremely complex very quickly. Definitely not something i would use as my normal design methodology. Im surprised they are so casual about something so problematic as maintaining data consistency.
@@gppsoftware These things though are both part of, literally, the definition of a microservice. You don't share databases and each service is aligned with a bounded-context. So in your example, it is fine for you order processing service to have a reference to a customer, but order processing is a different bounded context to customer management, so the nature of "customer" is different in each of these contexts, My guess is that all you need to represent a "Customer" in order processing, is a customer id or similar, enough to register the idea of "This Specific Customer" so that you can ask the Customer Service for any more detailed Customer information that may be required, this de-coupling is really the hole point. So what you are really saying when you say "the vast majority of microservices I have seen actually share a common database and data model behind the scenes" is "the vast majority of microservices I have seen WEREN'T microservices" which is true for me too. Most teams get this wrong!
@@gppsoftware The problem here is what do the names that we apply to things mean if we ignore the definitions that go with them? What is the use of calling something a microservice if it doesn't meet ANY of the criteria of a microservice, at that point it is merely a "service", and a poorly designed one at that. I agree with you that the practice is to not build microservices, but I don't think it helps to accept calling them microservices if they are not all of these things: independently deployable, aligned with a bounded context, Decentralised data management, Loose coupled.
@@gppsoftwareWell that is part of the definition of "bounded context" too, Eric Evans says that you should ALWAYS TRANSLATE data that crosses between bounded contexts so that you can 'adapt' (as in ports and adaptors) the information that flows across these critical boundaries.
Good points. Caching and messaging work well as services for micro services.
great explanation, any project you go you will be working with API so this is essentials things to know.
Thank you sir. I want to know what are the current issues in distributed computing
Recently come across your channel and I already feel like you've been living in my head for the last 10 years! Great videos. I haven't heard or seen (yet) you talk about consumer driven contract testing (like PACT) when working with micro services, do you have a video on it? I feel it adds a lot of value in the space!
I talk a little about that in a few videos, I talk about the principles here: kzhead.info/sun/ltSPfN6boXV4YKs/bejne.html I probably should do a video more specifically focused in this topic though, thanks for the suggestion.
@@ContinuousDelivery Thanks for the reply, Ill check that out and look forward to the next. On another note, would love to see some Q/A live streams on the channel..
"focused on a task" could be being an backend of a web app or converting an integer to a string
Great video, short and clear. Thanks.
Thank you, you are very knowledgeable. Decoupling Autonomous Independently deployable Easily removed without coordinating with others
Thank you
Think the discussion about bounded context is relevant when implementing an MS.
Really clear explanation thank you a lot sir!
Most welcome!
Great insights and food for thought. Thank you Dave.
Over the years I've seen many people making pretty abstract pictures of systems that are build up from loosely coupled independent modules, but I have never seen it actually work in the real world. In the real world, users want complex screens that gets data from all over the place with complicated rules and calculations, living in different modules and all of this needs to deliver results in a short time frame. And for all the talk of keeping your architecture clean, there is often just too much pressure and too little time to consider these factors. In the end, you always end up with a tightly coupled spaghetti mess that is twice as hard to maintain than if you just did everything in one project. Maybe there are people out there that somehow managed to do it right, I just haven't ever met them.
Great, first video seem that clear my Microservices Concepts in depth.
Graduating college and entering the workforce. University, by no means is useless, but I feel has not prepared me for a lot of the ‘ecosystem’ around development. Courses focus on languages, algorithms, data structures, operating systems, but never any courses on real-world problems, and insight from dealing with those problems. These videos are invaluable to me as I start the path of actual software engineering! Not to mention being so much more digestible. I just saw a video called “Why I don’t use else clauses”. I respect the hustle, but it’s refreshing to get real advice in a legible way from someone who isn’t a year older than me.
Don’t feel the college courses are useless at all. I am a 20 year veteran and I my time languages and frameworks have come and gone, but at the end of the day the base that I got in my CS degree with algorithms, data structures, compiler theory etc has not changed. The change is in how you do things. So that college edu is super useful over the long term.
Great video! Thanks for the quality content!
Very nicely done... thank you.
You are amazing . Learning lots from you.
Great Video. I didn't get your point on microservices and ports and adapters. Could you provide more details on it? Thanks
Part of the definition of a microservice is "aligned with a bounded context". The idea of "bounded context" comes from Eric Evan's book "Domain Driven Design". Here is Martin Fowler's definition: martinfowler.com/bliki/BoundedContext.html Eric says that when information "crosses between bounded contexts" it should be translated. You shouldn't share information "raw" between different bounded contexts. So for a Microservices, you should translate your outputs into a clear form, that you hope to be able to support in future, even when your service changes, and you translate your inputs from whatever the sender defined, into something convenient that you will use internally in your service. This gives you a bit more freedom, in both directions, for things to change without breaking your code. The best way to organise these translation points is with a "Ports & Adapters" approach: en.wikipedia.org/wiki/Hexagonal_architecture_(software)
I think micro-services for most organizations just turns into a versioning nightmare
Thank you very very very much for the lesson ! 👍🏿👏🏿
They have their place, but "loosely coupled" is often a panacea in an enterprise. The difference between poster-boy microservices and those typically found in an enterprise, is persistent data, and the fact that the organisation wants to treat services (and their underlying data models) as autonomous and loosely coupled one day, and as one cohesive data model the next. When someone demands a report that spans service data models, they don't want to hear about artificial boundaries. When the domain changes, it often requires API changes across multiple services, similar to adding a new field on a screen, which ripples through all layers of the architecture, all the way down to the database. Microservices, just like "components" of old, are typically best split along the seams between different levels of abstraction in the architecture.
You need to start with the business process and identify the tasks and events that trigger the processes. Micro service should map to the discrete tasks in a business process . An example would be the booking service for a hotel reservation system
Thanks man that's really a confusing idea I've struggled with. It's not easy to differentiate between SOA and microservices. I think it would be very interesting if you give examples of implementing microservices wrongly and explaining what is wrong with it. Wish you all the best thank you again
I have the same confusion.
Great Video! very clear in the information presented.
Thank you!
Great video. I’ve found one of the biggest problems with the actual implementation is modelling transactional consistency boundaries and sharing data. This is where I’ve found teams can really get tripped up. Do you have any videos or advice on these?
Indeed, this is a tricky problem. The SAGA pattern is designed to solve it. I have not actually seen it implemented within my organization, but I'm sure there are plenty of companies that are effectively utilizing it.
Excellent video. Have passed to my team for review. Please keep up your great work.
Thanks, will do!
Thank you for the very nice description of microservices principles. Can someone elaborate on what is ports and adaptors and why it is a counter example in the bounded context? To me the translation of internal to external representation is equivalent to adapter.
Thanks. I talk in a bit more detail about "ports & adapters" here: kzhead.info/sun/eLeBnplrpKt4a3A/bejne.html
Essentially he is advocating contract testing to ensure interfaces aren't broken. It would also be good to abstract out discovery. security, monitoring tasks into a common service mesh layer and let microservices focus on business rules implementation.
Would love to learn more about high performance computing with event driven architecture! Sounds really engaging.
I have a few videos that you may find interesting: Reactive Systems - kzhead.info/sun/mLaxfMqNgH-AqqM/bejne.html How Fast is Your Computer? - kzhead.info/sun/Y9aefbqfpWihhac/bejne.html The Next Big Thing in SW Architecture - kzhead.info/sun/YK2TnLOImpuJn2g/bejne.html
That whole bit at the end about defending interfaces and changing them rarely. My experience is that going heavy handed early on in a project with pact for example takes an inordinately heavy toll on development speed.
Great content! How would you call a service that is small, focused on one task, aligned with the bounded context, but is not independently deployable or autonomous? That is, they are developed by the same team and part of the same pipeline as the rest of the services? It seems this is the real value many organizations look for in the teachings of "microservices", but since there doesn't seem to be a name for them, they latched on to the name "microservices" because it's the closest thing to them. The services in the previous generation of SOA didn't follow these principles (specially the ones about aligning with the bounded context), so you can't call them "SOA services" either. How would you call them, to prevent terminology confusion in our industry?
The one big problem with microservices within business is the fact that whatever small autonomous program you are making probably has an open source solution out there already.
Hi. Great video as always. I work in Test management, and regarding your own experience; would you agree that a ‘challened’ team (or overworked, understaffed, mired in buggy code) working on one decoupled service/component, poses an over all risk to the whole system. If so, what you look out for as early warning signs, and what would be your guidance in such a case?
I think that you de-risk situations like this with tests. You test all new work and gradually test more of the pre-existing, untested, stuff close to what you are working on. You can improve the isolation of the code for a component of the system like you describe through good design, but I’d argue that the easiest way to get to that good design us also through testing - my argument is that testable code is better designed code too.
Thanks for the great video, one important aspect that you mentioned about, not requiring consumers to change something in order to change your service, which is broadly true, It would be great if you explain in depth different types of relations between the teams and the services they built, upstream vs downstream, and what are the best practices to governe such changes an interfaces, because in practice not all services are conforming to each other
I have several videos that cover those topics: "Why you SW Team Can't Scale" kzhead.info/sun/o9tvaJqHsJenoK8/bejne.html "Platform Teams" kzhead.info/sun/kt6BZ7iBj5WgeKs/bejne.html "Evolutionary Design" kzhead.info/sun/a6uIftpuoXypemg/bejne.html and many more.
This is gold! Very well done sir.
Great content sir.