The Hidden Cost Of GraphQL And NodeJS

2023 ж. 19 Қаз.
176 158 Рет қаралды

Recorded live on twitch, GET IN
/ theprimeagen
Reviewed article: www.softwareatscale.dev/p/the...
MY MAIN YT CHANNEL: Has well edited engineering videos
/ theprimeagen
Discord
/ discord
Have something for me to read or react to?: / theprimeagenreact
Kinesis Advantage 360: bit.ly/Prime-Kinesis
Hey I am sponsored by Turso, an edge database. I think they are pretty neet. Give them a try for free and if you want you can get a decent amount off (the free tier is the best (better than planetscale or any other))
turso.tech/deeznuts

Пікірлер
  • I mean, in an interview someone said that NodeJS use a single thread and the interviewer didn't believe it, a lot of devs think promises are multithreading

    @neociber24@neociber246 ай бұрын
    • That would be scary as heck. Imagine all the concurrency issues that would crop up, oh dear.

      @Fs3i@Fs3i6 ай бұрын
    • ​@@Fs3i couldn't they? If functions do not alter a global state then they should be able to be parallel, right?

      @sergisimon9620@sergisimon96206 ай бұрын
    • If your going to go async you might as well go multi thread... so I would too expect it to be multi threaded.

      @chainingsolid@chainingsolid6 ай бұрын
    • @@sergisimon9620 That's a big if. It's very easy to alter global / shared state, and then you deal with race conditions.

      @Fs3i@Fs3i6 ай бұрын
    • Yep. I had that happen this week. A dev thought that "async / await" was multithreading. Scary.

      @aeggeska1@aeggeska16 ай бұрын
  • Senior JavaScript developer had resolved this already - he had callback hell, then rewrote everything to Promises. Await came out, and then rewrote back to Promises. Then rewrote everything back to callback. *Because it is just more comfortable* 😅

    @MarisSamats@MarisSamats6 ай бұрын
    • The second sentence should have "await" at the end.

      @0xDEAD_Inside@0xDEAD_Inside6 ай бұрын
    • @@nosferratu Eveeerythiiiiiing. I wish I could post the mem photo hahah

      @juliocesar-fc1nm@juliocesar-fc1nm6 ай бұрын
    • haha sounds like the video from "Programmers are also human"

      @kevinflorenzdaus@kevinflorenzdaus6 ай бұрын
    • Async/Await lesson curves. Make sure you put async in front of the function definition and AWAIT in front of where you want the code to stop until the await happens... Cool.. Then you are debugging the transpiled code. HOLY F... Look at this... callback helll...

      @wbtittle@wbtittle6 ай бұрын
    • Senior JavaScript developer learned one of Java, C#, Go, or Rust and implemented his server-side code in a language more appropriate for the problem.

      @rossbagley9015@rossbagley90156 ай бұрын
  • Thanks man , this helps a lot . We always got level up after watching your videos and you provide very good quality content . Anyone can create a video reading a blogpost but you discuss about it very well , explains the topic by keeping beginners in mind and taking about pros and cons of it . I rarely write comment but this is worth it 👍

    @user-iv5sl2dn3r@user-iv5sl2dn3r6 ай бұрын
  • JavaScript will try and run async code as sync code until it hits a point that forces it to the back of the event loop. Meaning if you attach the async keyword to a function with no await keyword inside it, it will run in sync and never be placed at the back of the event loop.

    @BlackAsLight448@BlackAsLight4486 ай бұрын
    • A different way to say it is that it is cooperative multitasking. It only switches when the user code tells it to switch. It won't pre-empt that switch.

      @mattymerr701@mattymerr7016 ай бұрын
    • Not sure if I like it or hate it - it's probably good for perf but boy is it unintuitive sometimes.

      @huge_letters@huge_letters6 ай бұрын
    • This is an extremely important detail, thanks for pointing it out. In the case of async functions which cache their response, the overhead is the JS engine creating a Promise object. Even if you write your code using async, if it can be executed in sequence without deferring to to the task queue, then it will be executed in sequence. Prime hates JavaScript, so he doesn't learn it and he doesn't understand it but he talks a lot about it...

      @fardelian@fardelian6 ай бұрын
    • @@fardelian Prime understands JS very well and has been writing it professionally for years, in performance sensitive environments. He can still have misconceptions but saying that he doesn't (want to) learn it is absurd.

      @huge_letters@huge_letters6 ай бұрын
    • Not quite. If you return a promise form a function and don't await it, the promise will be placed on the back of the queue. So the rest of your code can't depend on the result of the promise. If you don't await the promise it's going to be a pure side-effect. Kinda useless, might be better to use background workers that run in a separate thread

      @spicynoodle7419@spicynoodle74196 ай бұрын
  • To be fair, if SQL was actually SLQ, then it could be pronounced 'Slick' which I'd be a bit more tempted to use in speech than 'Squeal'

    @tharsis@tharsis6 ай бұрын
    • *Sequel

      @0xDEAD_Inside@0xDEAD_Inside6 ай бұрын
    • or “sleek”

      @EricSampson@EricSampson6 ай бұрын
    • @@0xDEAD_Inside *esquill

      @vcokltfre@vcokltfre6 ай бұрын
    • yeah it's sequel not squeal, idk why he pronounces it like that

      @FlanPoirot@FlanPoirot6 ай бұрын
    • Call it squirrel 🐿️

      @rafaelbordoni516@rafaelbordoni5166 ай бұрын
  • Bad programming is bad programming. Using callbacks can lead to callback hell with bugs and impossible refactoring strategies, while Promises lead to event loop starvation and slow performance. But honestly, I can deal with a misplaced async situation much more quickly than the callback hell.

    @iulianbojinca7309@iulianbojinca73096 ай бұрын
  • You man, i discovered your videos just few weeks ago, and i wanna thank you. I'm quite hardened SDET and working with JS|TS more than 5 years (and few years on other langs before), but most of the time i worked as a lead on the-only sdet on a project, and i never had a buddy, with whom i can discuss js-related stuff, more to say, really deep-knowledged buddy. And your videos for me - it's a great pleasure to just hang out, learn very interesting stuff which i presumably will never use in my work , but it expands my view and understanding of the lang and tools. i hate reading tech literature, so everything that i've learned - is just experimenting and trying to code what is in my head. and i think i'm not the only one in this situation, and i hope more of someone like me will find your content helpful So please, explain more, show more, rant, but show how do better. Thank you.

    @Elkemper@Elkemper5 ай бұрын
  • There's another cost: Not knowing computers or working together with other software ecosystems. I had to work together with another company, which based their GraphQL API onto a standard SQL server. They didn't know how to access the GraphQL-API without a framework (like the URL). They didn't know how documentation works at all. Their error handling was non-existing. Their database structuring was just one giant JSON and little thinking was done about how to access certain data points. I think the sole focus onto JS/TypeScript and resulting frameworks really hurts the knowledge of what computers actually do or how other software ecosystems should be able to interact with yours.

    @aotein@aotein6 ай бұрын
  • Promises are not inheritly slower, just the implementation. C++ promises can be optimized away completly. Rust's futures are optimizable as well. If JavaScript is your problem, you shouldn't change styles to make your code unmaintainable; you should change your language.

    @jaysistar2711@jaysistar27116 ай бұрын
    • surprised this isnt the top comment, this place is too full of frontend developers 😅

      @razhemo4191@razhemo41916 ай бұрын
    • @@razhemo4191 You gotta make sure everyone's included

      @Dozer456123@Dozer4561236 ай бұрын
    • The discussion was about JavaScript promises…

      @asdqwe4427@asdqwe44276 ай бұрын
    • @@asdqwe4427 why limit the discussion?

      @razhemo4191@razhemo41916 ай бұрын
    • @@asdqwe4427 " If JavaScript is your problem, you shouldn't change styles to make your code unmaintainable; you should change your language."

      @dhanushc5744@dhanushc57446 ай бұрын
  • Really good explanation we need to keep this in mind. I had one thought how are Observables and Subjects vs Promise

    @tanmaytiwari2450@tanmaytiwari24506 ай бұрын
  • 13:00 Hmmm not sure if I remember it right but I’ve tried this before, async functions that doesn’t perform an await operation does not get pushed back the queue, they act as though they aren’t async, the only time when the function gets pushed back the queue is when they do await.

    @aprilmintacpineda2713@aprilmintacpineda27136 ай бұрын
    • But when they return, they wrap the return value in a promise, so at least one microtask is added. I think the morale is to avoid using async functions if you don't need to. Async functions are fine for slow IO operations such as db queries.

      @SurenEnfiajyan@SurenEnfiajyan6 ай бұрын
  • What if you loop the fields in the querry but do smth like a querry builder so we only get 1 promsise for all of the required data?

    @asoftyn4805@asoftyn48056 ай бұрын
  • Funny that most of the issues pointed were not about graphql...

    @LucasLocatelli@LucasLocatelli6 ай бұрын
  • Ultimatly it all depends on what project you are working on! I have a preference for GraphQL over Rest

    @grimm_gen@grimm_gen6 ай бұрын
    • I think the point of video is graphql is slow doesn't matter your preference or the project. Its very easy to write bad code with gql/js.

      @dhanushc5744@dhanushc57446 ай бұрын
    • @@dhanushc5744 gql isn't slow, maybe apollo's implementation is slow, js is slow, but gql isn't slow. There are rust implementations of gql that are crazy fast, and don't have the same insane mem overhead.

      @deidyomega@deidyomega3 ай бұрын
  • JavaScript on the backend is only cost

    @PwrXenon@PwrXenon6 ай бұрын
  • You are correct that writing SQL is better than ORMs. Ted Neward explained the problem pretty well in his article, "The Vietnam of Computer Science".

    @thewiirocks@thewiirocks6 ай бұрын
  • Graph Kwil - You're a mad man

    @LurajerLP@LurajerLP6 ай бұрын
  • Thank you. This explains so many of my current issues...

    @wbtittle@wbtittle6 ай бұрын
    • Cut and Paste might be ahead of this... But when people cut and paste async await with tracing all over the place...

      @wbtittle@wbtittle6 ай бұрын
  • "When your tracking software slows down your entire universe, that's not good." Best description of the Heisenberg Uncertainty Principle?

    @JeremyKolassa@JeremyKolassa6 ай бұрын
  • The two selects he's referring to at 9:51 are the SELECT stuff FROM users WHERE id = ?, and SELECT stuff FROM items WHERE userid = ?. So he's saying that you can do that roundtrip in < 100 ms, which is true, but if it's only 10 000 items, then the ping-ponging to and from the db is a significant source of time. I remember when I did 500 inserts and THEN committed, vs doing 500 insert + commits, and how the whole thing went from seconds to milliseconds just like that. In my defence, I tried, saw bottleneck, and THEN made the batch inserts, which are a bit more complicated in that the code will look a bit different.

    @HrHaakon@HrHaakon6 ай бұрын
    • I wonder how big this application's access log is.. and how big the disk it lies on.

      @gwaptiva@gwaptiva4 ай бұрын
  • Since I mostly write code that is not extremely performance-sensitive, I have no problem using JS (or TS). In the day, I was a big C# user and appreciated values (integers, structs, etc.) didn't need to use the heap. Struct, in particular, is a useful way to reduce GC. I'm keenly aware of the cost of promises because, before promises, I implemented my own versions (in JS and C#) by creating an object to hold the pending state). How much are today's programmers aware of how these mechanisms are implemented? And why, for much code, readability is more important, but not always. As go graphql -- I've found SQL simpler, but that's another conversation.

    @BobFrTube@BobFrTube6 ай бұрын
    • Nowdays C# makes some kind of black magic with Task state machine and even more one with ValueTask. The latter is specifically optimised to have minimal perf impact on cases where value is returned synchronously.

      @Myself0094@Myself00946 ай бұрын
    • GraphQL and SQL have nothing in common, you can’t compare them…

      @gro967@gro9674 ай бұрын
  • These issues can be addressed based on the database, for example GraphQL has a graph structure, Neo4j is a graph DB, the Neo4j-GraphQL library transforms GrapQL queries into Cypher (SQL for graph DBs) queries, since Neo4j takes care of the whole query instead of GraphQL resolvers chatting, it removes the "chatty protocol" part, or the "n+1" issue common in GraphQL, though not sure how this could be fixed in non graph DBs...

    @CarlosEstebanLopezJaramillo@CarlosEstebanLopezJaramillo6 ай бұрын
    • N+1 issue can be solved easily by hiring a descend developer or using a good ORM if you can't find one.

      @blender_wiki@blender_wiki4 ай бұрын
    • @@blender_wikiIndeed, just saying Neo4j has the solution embedded in their GraphQL library, so I don't even need to think about it :)

      @CarlosEstebanLopezJaramillo@CarlosEstebanLopezJaramillo4 ай бұрын
  • What library would you suggest over express? I've always thought of express as the standard go to and never questioned its performance. I mean everything I've ever used it for seemed fast.

    @Kurimson@Kurimson6 ай бұрын
    • Yeah that's what I am thinking. I got a nice React, Apollo and Mongo stack going on here. It's a great package to bring stuff to market and try things out.

      @maltbiscuit9995@maltbiscuit99955 ай бұрын
    • It is the standard for node but node sucks. Poor performance, fake types, and has no backend ecosystem.

      @jeffreysmith9837@jeffreysmith98375 ай бұрын
    • Nestjs with fastify. If performance is that serious, golang. If you aren't having performance issues then don't worry about it 🤷 Instagram runs on Python so its not that serious

      @AmatuerHourCoding@AmatuerHourCoding2 ай бұрын
  • looks like 90% of people who learned graphql use it completely wrong! You should never do that for each loop and fetch each element of the array from IO! What you should do instead is Retrieve the query object the user sent, check if the array field is not undefined, and then build an SQL string that is performant and fetches what the user exactly requested. JUST CALL THE DB ONCE NOT A DOZEN TIMES

    @ES-eb6pb@ES-eb6pb6 ай бұрын
    • And 99% don’t need to be using graphql at all

      @asdqwe4427@asdqwe44276 ай бұрын
    • @@asdqwe4427 that's like saying people shouldn't be using rest. it's just a standard, low understanding comment

      @bilbobaggins7075@bilbobaggins70756 ай бұрын
    • @@bilbobaggins7075 graphql is hard to get right. The flexibility is really cool if you have tones of different consumers. But most people have one frontend and then it’s way simpler to make simple api that caters to the use case the that you do have. Don’t make some generic solution that’s hard to make scale.

      @asdqwe4427@asdqwe44276 ай бұрын
    • Not possible when nested objects with paging is involved. That `... FETCH NEXT xyz ROWS ONLY` really kills it.

      @svetlinzarev3453@svetlinzarev34536 ай бұрын
    • ​@@bilbobaggins7075no it's not... GraphQL is not a standard, it's a spec. GraphQL does not conventionalize anything.. There is no reason graph QL has it's own query language instead of just being a REST API accepting JSON ASTs..

      @gabrielspeed5464@gabrielspeed54646 ай бұрын
  • If your application is at the point where you're concerned about a core language feature slowing things down, maybe you shouldn't be using JS on the backend at all. GraphQL is a pleasure to develop with and I will happily trade the 1% of extra overhead for how much easier it makes frontend dev without having to make a bunch of backend changes every time you work on a new feature.

    @dylan_the_wizard@dylan_the_wizard6 ай бұрын
    • if it only was 1%...

      @jordixboy@jordixboy6 ай бұрын
  • That last rant about that code snippet was one of the most important things I think I've ever heard.

    @mosescosme8629@mosescosme86293 ай бұрын
  • I was conducting some interviews in my previous job. I had a few questions to go over to see how well they understood js. Most of them couldn't tell me how you can perform async tasks without promises. Callbacks are a forgotten technology for most devs in their 20s.

    @IhsanMujdeci@IhsanMujdeci6 ай бұрын
    • Most coders are not coders, without a framework they are lost

      @JacobBogers@JacobBogers5 ай бұрын
    • @@JacobBogers you're using Dart complaining about frameworks on a topic where frameworks aren't even involved (promises are native in JS) You don't even want to use Kotlin and Swift directly. The irony here is so sweet

      @gabrielspeed5464@gabrielspeed54645 ай бұрын
    • ​@@gabrielspeed5464 "you're using Dart complaining about frameworks" That is very trolly and dishonest claiming things I never even implied let alone said. But what to expect from an arrogant lazy POS, who cant even code.

      @JacobBogers@JacobBogers5 ай бұрын
  • That one time I used graphQL (unwillingly), after 4 months into development, I resigned because stakeholders didn't believe it was the bottleneck.

    @yowwn8614@yowwn86146 ай бұрын
    • did you work at a startup?

      @justADeni@justADeni6 ай бұрын
    • Graphql is only the bottleneck if you (or someone on your team) wrote a shitty gql server. Same with client side code. You can write a gql server in rust, with blazing fast speeds and none of the mem overhead. You can consume that graphql without any library, if all the libs are too slow. Or change your client lang. Write your gql client in rust, compile to wasm, then expose the entire thing to js as the glue if you need. gql is just a language, nothing in the language requires slow code.

      @deidyomega@deidyomega3 ай бұрын
  • Maybe I'm misunderstanding something basic here, but isn't all async code basically the same under the hood? Callbacks will also get pushed to the tail of the event loop and actually get executed whenever is their turn, won't they?

    @gonzalomunoz2767@gonzalomunoz27676 ай бұрын
    • Promise object has an overhead, idk the details

      @jeffreysmith9837@jeffreysmith98375 ай бұрын
  • Callbacks as continuation-passing style (CPS) + sane compiler = Callback heaven (ref: tail call optimization and cps basically turns unbounded recursion -> iteration) Callback as CPS mechanism + no actual compiler = callback hell

    @jimhrelb2135@jimhrelb21356 ай бұрын
  • The individual promise thing is definitely not sensible in the graphql context. It would be a bit different if it was like a component system where you want state for something as low as possible, so you might have many components making similar calls for different data that can be batched into a request. Where doing the extra request in a single promise from the top could be impractical or increase dataloading.

    @thekwoka4707@thekwoka47076 ай бұрын
  • If it's about optional async function it doesn't work as you've mentioned at least in Chrome browser. I was a little surprised this some time ago when I've found that async doesn't equal setTimeout(cb, 0). async function test() { console.log(1); }; test(); console.log(2); Will return: 1 2

    @SlawomirBudzynski@SlawomirBudzynski5 ай бұрын
  • is there any evidence which confirms that promises-async/await are slower the callbacks?

    @TheRcfrias@TheRcfriasАй бұрын
  • I avoid chatty protocols by just anticipating every action a users going to take before they even enter my site and serve them everything in the database in one request and make every possible change they might make at once before returning. Can't have a chatty system if you only use 1 request.

    @dabbopabblo@dabbopabblo6 ай бұрын
    • No joke we actually did this for a client once. On site load, download the entire user's data locally. For our very weird use case, it actually ended up being faster.

      @deidyomega@deidyomega3 ай бұрын
    • ​@@deidyomegaThat's crazy, I'm rewriting an app that did not only exactly that but also leaked every other user's full data in every response from the backend (including their [encrypted] passwords!!! You could see it all by just hitting F12!) Changing that, on top of the UI changes my client wants, would require a rewrite of 90% of the codebase anyways so I'm redoing this horseshit in go+htmx+templ now lmfao

      @chunkspiggle3916@chunkspiggle39163 ай бұрын
  • "Are callbacks really async if your date never does them?" He got me

    @AqgvP07r-hq3vu@AqgvP07r-hq3vu6 ай бұрын
    • I mean thats definitely not in sync, or else there would be a callback. from your date.

      @ark_knight@ark_knight6 ай бұрын
  • What is the name of the drawing software he was using?

    @user-vl4gh7pt3d@user-vl4gh7pt3d6 ай бұрын
  • If you care a lick about performance, why the hell would you use JavaScript in the first place?

    @anlumo1@anlumo16 ай бұрын
    • Combine that with object prototype pollution and eval exploits it is just not a good language for server side at all. (And I like JS in the browser) It might even be worse than using PHP - if it has filesystem access.

      @etherweb6796@etherweb67966 ай бұрын
  • do a vid where you optimize the code you showed at the end

    @JarrodNotJared@JarrodNotJared6 ай бұрын
  • Interesting, i was wondering why nodeJS is 3x slower than PHP with swoole, despite the fact that it was built for async first, and async behavior in PHP was very after the fact. That's some serious computational pork!

    @neptronix@neptronix6 ай бұрын
  • 14:35 what site is that?

    @alex_nita@alex_nita5 ай бұрын
  • 20-years in the industry and all I have to say is, "I told you so".

    @rabbitcreative@rabbitcreative6 ай бұрын
  • Are Promises without using the async and await keywords still problematic?

    @maplepenguin8568@maplepenguin85686 ай бұрын
    • Yes, async await is just syntax sugar for promises. As he described every ".then()" is incurring the costs he's describing as well

      @pkop4@pkop46 ай бұрын
  • Do JS promises really continue asynchronously even when the promise is resolved immediately? I'm not a js dev (thankfully) but when I've actually needed that behaviour in the past I've had to use a setTimeout(0), even with promises.

    @georgehelyar@georgehelyar6 ай бұрын
    • When you resolve the promise immediately, the callback passed to then() method is not called immediately. Promise.resolve('async').then((v) => console.log(v)); console.log('sync'); this code will first print 'sync' and then 'async'

      @SurenEnfiajyan@SurenEnfiajyan6 ай бұрын
    • They don't "continue" they pause and cede control to the main loop, before being restarted when they hit the top of the queue. It has the advantage of stopping the processing getting lumpy at the cost of performance. i.e. on average responses take a little longer, but none egregeously so. Lumpy (jitter) data processing is painful because it shows with the frequent short messages taking way too long, rather than with the occasional long running culprits. ( it's analogous to a sleep(0) if you are using a real-time micro-kernel.)

      @outwithrealitytoo@outwithrealitytoo2 ай бұрын
  • “My favorite type of tweet is the one where everybody loses.”

    @rodrigolj@rodrigolj6 ай бұрын
  • 13:27 - is that true tho? When file write is done the callback would still get added to the task queue, no? Otherwise it would break code execution if there're other tasks running at the moment - it still has to wait until at least the current task is finished until executing a callback. Otherwise if say I'm executing task1, register a callback for my file write, task1 gets finished, file write is still executing - does that mean now it won't just start executing task2 and will wait until write finishes to call my callback? So I don't really understand the difference here between a promise and a callback - unless what you mean is that callback will get added to the start of a queue and will be executed as a next task but the promise will get added to the end of queue and might wait for quite a while until it gets its turn.

    @huge_letters@huge_letters6 ай бұрын
    • The difference comes down to the user agent. Promises are microtasks, just like callbacks. But callbacks can be queued differently, depending on what kind of callback it is. In Node there are callbacks for different phases of the event loop; timers, pending callbacks, poll, check and close callbacks ( socket.on('close', () => {}) As you can see there isn't one place where a callback is called, but it depends on the callback and the current phase that the event loop is in. This is the macrotask queue. Promises (and process.nextTick(), queueMicrotask) are executed in between these phases. This is called the "microtask" queue. Microtasks have higher priority and are always executed before the next macrotask in the queue.

      @dealloc@dealloc6 ай бұрын
    • @@dealloc I see, but since microtasks are executed before the next macrorask and promises are microtasks - wouldn't that mean they would be executed before callbacks(which would go to macrorask)?

      @huge_letters@huge_letters6 ай бұрын
  • Sorry but how aren't promises just callbscks ? I always thought they were the same kust written different

    @pedroaugusto656@pedroaugusto6566 ай бұрын
  • Wait I wanted to use NodeJs and Graphql in my project but I am confused now can anyone tell me how you supposed and prefer to develop a large maintainable and easily scalable backend for something like social media platform like facebook ( for very start )

    @sideone3581@sideone35813 ай бұрын
    • Rest API first finish your app. Slowly migrate to graphql if necessary if not stay in rest API

      @fcnealvillangca7943@fcnealvillangca79433 ай бұрын
    • You don't. Nobody builds the app that will eventually have 100 million users from day 1. Use whatever gets it done until you have money coming in. Facebook employs literally thousands of engineers; you can't just have what they have from getting one weird trick off the internet. "Easily scalable" is for teams working on new products for existing large companies.

      @Nassifeh@Nassifeh2 ай бұрын
    • @@fcnealvillangca7943 I am thinking about gRPC now , I am a dumb : I don't know any

      @sideone3581@sideone35812 ай бұрын
  • Arent promises resolution microtasks? In the video he illustrates that they are tasks

    @aelsayed313@aelsayed3136 ай бұрын
  • it is incredible that this channel monetises reading blogs.

    @hanabimock5193@hanabimock51936 ай бұрын
  • what would be the criteria to make a function async?

    @jordixboy@jordixboy6 ай бұрын
    • when you want to run a blocking code like Db/network calls etc

      @dhanushc5744@dhanushc57446 ай бұрын
  • Wouldn't be EFCore (ORM) with LINQ be a 'slq'???

    @satyayuga0@satyayuga05 ай бұрын
  • The rant about "x" reminds me of Yahzee (of Zero Punctuation) calling tweets "excrements" now 😅. Epic video Prime

    @mattius17@mattius175 ай бұрын
  • Why would you ever return more than 100 items to the client from an API?

    @Pekz00r@Pekz00r5 ай бұрын
  • I wonder if this dataloader promises issue also affects django + graphene

    @flamendless@flamendless6 ай бұрын
    • Oh it does. Badly. Every time I have to debug the implementation’s ‘promises’ port to Python, it feels like getting hit in the head.

      @pbkobold@pbkobold6 ай бұрын
    • @@pbkobold so more or less it has something to do with the implementation of async (event loop)? In my case we use gunicorn. I'll read about the implementation of that

      @flamendless@flamendless6 ай бұрын
  • So, isn't the conclusion to use a more suitable server-side language for GraphQL/REST/complex business logic/everything on the server side? Unless you're trying to bang out some sort of prototype or tiny one-off site...

    @rossbagley9015@rossbagley90156 ай бұрын
    • The lesson is, I believe, use technologies that allow you to do more work in the database. What is killing Node here is the chatty protocol, but the chatty protocol is the primary problem. This leads Node to bow out early in the scaling game but alternative solutions based on the same protocols only scale in the sense that "you can pay for double the compute and ten times the network capacity than you could if you didn't use naive database abstraction libraries, but it will at least work (unlike Node)". Node is a canary in the coal mine for the underlying issue.

      @outwithrealitytoo@outwithrealitytoo2 ай бұрын
  • Or you can use java virtual threads, which are simple -- i.e. no promises, no callbacks; no async/await, and get all the scalability benefits of microthreading. Instead of constructing C so can give it to B as a promise or callback, then give B to A; in java you just do A,B,C.

    @freeideas@freeideas6 ай бұрын
  • 6:46 i wonder how Graphile would perform in comparison

    @magfal@magfal6 ай бұрын
  • hey man i've always loved your content, but i think your reasoning about promises is wrong, that is the bases of concurrency (sharing the resources instead of hoarding them), you would have the same issue with callbacks (even if things are not actually async, you would stil steal the resource while doing your sync stuff and other requests would pay the price, you just moved the issue to somewhere else, not fixed it). my guess is (from what i've notices) that when promises are already resolved (the cache you mentioned, but also it can happen when reading a huge directory without throttling ) the list (which im pretty sure is an array) becomes too big and inefficient for pushing/popping elements from it. await also counts as 2 promises either way (each await), thats why `await null` does not throw, because you are technically doing `await Promise.resolve(null)`

    @pedropertino7324@pedropertino73246 ай бұрын
  • Aren't Promises just roided out callbacks? Like Object-Oriented callbacks. Except roided out ends up meaning way slower

    @lionbryce10101@lionbryce101016 ай бұрын
  • Maybe stop using foreign keys to join nested items with a parent data model. Instead, capture the parentID in the item and find all items by the patentID field, and directly query the items. No need to query the user just to get the users' items. Doing this then removing .populate('userItems') from the user resolver will speed up the server respons. My limited experience and this observation tells me to avoid joining on nested lists. I had a query contacting a resolver that had. populate('userItems') the GraphQl query wasnt requesting the items and ther server wasn't sending the item. The .populate still causes your server traverse the data and smow its response to the client.

    @wildbuffalo1712@wildbuffalo17126 ай бұрын
  • Isn't this solvable using Dataloader?

    @StingSting844@StingSting8446 ай бұрын
  • «GraphQL gives you a promise of easy...» But the promise part is hard, it adds some CPU overhead, it seems.

    @jerrygreenest@jerrygreenestАй бұрын
  • Am on the team write SQL instead of using ORM. use ORM if performance not involved

    @adesanoyesamson668@adesanoyesamson6685 ай бұрын
  • Isn't this basically the n+1 problem at an API level?

    @renatocustodio1000@renatocustodio10006 ай бұрын
  • 1:00 I wanna learn how to make async without promises! Can anyone recommend me some article about this?

    @mble@mble6 ай бұрын
    • btw async functions implicitly return promises. so you still do .then or whatever

      @dhanushc5744@dhanushc57446 ай бұрын
  • NodeJS threads comes from making a another request to another NodeJS instance. Don’t do all the the work in GraphQL resolvers if that’s slow, feature flag, auth etc

    @fredrik8737@fredrik87376 ай бұрын
  • Meanwhile me happily using threads, async and such noice stiff like this in c# without any drama

    @idkidk9204@idkidk92046 ай бұрын
  • If express is that bad as a framework, what do you suggest us to use instead? In node.js ecosystem of course.

    @elomatsouagboka6844@elomatsouagboka68445 ай бұрын
    • nodejs sucks balls

      @jeffreysmith9837@jeffreysmith98375 ай бұрын
  • There's so many things wrong with the approach I don't even know where to start

    @Ked_gaming@Ked_gaming6 ай бұрын
  • "Functions are people" -- primeagen, probably

    @Saiphes@Saiphes6 ай бұрын
  • So, the fix is to implement a new task keyword that does real threading to functions in JavaScript.

    @jfftck@jfftck6 ай бұрын
  • "But GraphQL is complex!" Use Postgraphile "But the SQL it generates is slow!" Use Postgraphile "And avoiding N+1 in resolvers is hard!" Just use Postgraphile

    @mileselam641@mileselam6416 ай бұрын
    • But then you would be programming everything in SQL with Postgraphile and putting business logic in SQL. Also anytime you want to make a tiny change to your function, you would have to create a new migration file and make it hard for a version control to show what's actually changed.

      @varshard0@varshard06 ай бұрын
    • @@varshard0 Postgres supports idempotency in DDL. CREATE OR REPLACE FUNCTION foo() … Put that in an SQL file and re-run as often as you need to. Always ends up in the same state. Structural changes to tables is harder, but that's no free lunch in any workflow. ALTER TABLE IF EXISTS foo ADD COLUMN IF NOT EXISTS bar int4 NOT NULL DEFAULT 0; Fast for adding the new column. Free when the column already exists. Idempotent SQL. There's your diffs and version control. DB schema *is* business logic. Algorithms have always taken a lower priority to data structures among top coders, and databases are nothing if not data structures. "But the API shouldn't leak the DB schema." 1. 99.9% of the time, the API and schema are the same, especially with most ORMs. 2. You can always just make a new schema and define views to reflect your API. 3. Defining the DB schema bubbles up so you don't have to redefine records and types again at the API server level and yet again at the UI consumer layer. Starting in the middle of your stack compromises the whole stack; your DB structure gets compromised by the impedance mismatch, and your development cycle gets compromised by writing all your types manually in triplicate. You're right that Postgraphile introduces some complications to typical dev workflows, but let's not pretend our current dev workflows are smooth and seamless as they are. Better to err on the side of clean and uncompromised data structures. But if Postgraphile isn't to your liking, there's also Hasura. Point is, life's too short to be writing object definitions in triplicate only to hamstring you DB's performance. Focus on the DB structure, and you've fixed most performance and workflow problems before they even start.

      @mileselam641@mileselam6416 ай бұрын
    • @@varshard0 "Bad programmers worry about the code. Good programmers worry about data structures and their relationships." - Linus Torvalds

      @mileselam641@mileselam6416 ай бұрын
  • Sarcasm so strong I can never surely say which technologies he works with..

    @MuhammadTajammulZia@MuhammadTajammulZia3 ай бұрын
  • Oh no, I might get a 5ms delay if I'm getting nested data for 100 users. Shucks, I guess I should optimize that when the db queries are taking up 400ms and the network latency is 100ms

    @jackdanielson1997@jackdanielson19976 ай бұрын
  • I missed the reference/joke about asking javascript devs to not write promises and them being mind blown 😬 can anyone explain please

    @aybmab2@aybmab26 ай бұрын
  • At this point programmers don’t really know how anything they are coding works behind the scenes.

    @natedavidoff668@natedavidoff6686 ай бұрын
  • you gotta stream an sc2 session sometime dude

    @voidreact@voidreact6 ай бұрын
  • Omg, they lost me at not wanting to use JOIN in SQL by default but separate calls. I honestly start to think abstractions, styles and such were all bad ideas as they are "guidelines over thinking"... but one cannot really avoid thinking otherwise result will be bad....

    @u9vata@u9vataАй бұрын
  • Squeal got me every time 😂

    @mateogalic7128@mateogalic71286 ай бұрын
  • yet another 'I built a benchmark that shows.....writing with chisels on stone tablets is faster than Rust, Go, C, and Commodore Basic"

    @W1TEA@W1TEA6 ай бұрын
  • Orms are like graphql - fast to get a prototype running but with massive tech debt later on in mature software

    @overbyte@overbyte2 ай бұрын
    • Controversially I'm going to say "it's not a speed thing, it's a skill thing". With ORMs/Graphsql you can get a naive prototype running with people with a light understanding of data and no SQL. However, as the transaction number and data volume increases, and as requirements change requiring data migrations occur; so it is that staff with greater knowledge of data are required. When it is time to scale, if you start with skilled developers and don't go down the data-framework rabbit hole then all you need is a bolt on DBA. If you went down the rabbithole then a) your devs have probably taken flight b) you're new staff are going to have the joy of migration c) a DBA won't be able to help much until the code is sorted. Sometimes "time to scale" is just moving from dev to prod ... insert burning house meme here

      @outwithrealitytoo@outwithrealitytoo2 ай бұрын
  • 19:30 heh, I typed that :P

    @maxlife459@maxlife4596 ай бұрын
  • I think the problem here is using js on the backend, not as much graphql

    @worgenzwithm14z@worgenzwithm14z6 ай бұрын
  • Even the people who made Express know how bad it is

    @andreicalegari1837@andreicalegari18376 ай бұрын
  • What's hidden about it?

    @seancooper5007@seancooper50075 ай бұрын
  • As a node dev, I don’t claim anyone who prefers promises to async/await

    @Brian-ro7st@Brian-ro7stАй бұрын
  • The Structured Languages for Queries (SLQ) should of course be pronounced as "slick" ;)

    @malcolmhutchison@malcolmhutchison6 ай бұрын
  • The resolver doen't need to run several call per field. It can one query instead. Building own GraphQL endpoint is a waste of time. It's simpler and faster to use Hasura GraphQL server.

    @hipertracker@hipertracker6 ай бұрын
  • I jumped on the async bandwagon and was ALL about it. EVERYTHING was async. Then my project got large, and the mental debt was so significant. ONLY use it when you truly need asynchronous code. It doesn't need to be and shouldn't be all or nothing. For example, if you are writing logs to a file, that can be async to not block the app's execution, which is a good practice. However, using async for every operation, especially in larger projects, can lead to issues like race conditions and increased mental overhead. Therefore, it's essential to use asynchronous code judiciously, where it provides a real benefit, and not make everything async 'just because.' This balanced approach ensures a more maintainable and efficient codebase.

    @clamhammer2463@clamhammer24636 ай бұрын
  • Server side JavaScript was always a bad idea

    @billybob1620@billybob16203 ай бұрын
  • by the time they fix this there'll be another framework out where this isn't even a problem

    @lulagoodwin5372@lulagoodwin53726 ай бұрын
  • "the hidden performance cost" plot twist - it's not even hidden

    @MyGroo@MyGroo4 ай бұрын
  • With the memory thing, this is why garbage collection gets such a bad rep. It's not that garbage collection is slow. It's creating all that garbage in the first place that's making things slow! The garbage collection part is just where you end up realising the cost. Kinda like blaming some code for dereferencing a nullpointer when it shouldn't have been passed one in the first place.

    @hannessteffenhagen61@hannessteffenhagen615 ай бұрын
  • The hidden cost of NodeJS? Your soul.

    @IS2511_watcher@IS2511_watcher6 ай бұрын
  • The problem in resolver not in graphql… you should write it like that, so your resolver end up with single mega sql query that will be executed once. In single request doing more than 1 query is the most stupid idea…

    @MrMassaraksh@MrMassarakshАй бұрын
  • Oh, I get it. If you write garbage code, the garbage collector has to work harder.

    @felipedidio4698@felipedidio46986 ай бұрын
  • all points were fair but I would like to add a point also there's a difference btw const promises = []; for (let i = 0; i < 5; i++) { promises.push(IOCall()); } Promise.all(promises); and for (let i = 0; i < 5; i++) { await IOCall(); } just think about, what's could be deference here, so it depends on what kind of application you are making. Should understand the internal implementation. I did 5,000 RPS s3 calls of around 30kb json size, to put data not get. with just 100mb of RAM express server on t3.micro inside docker, including APM on ECS Just think about it. I did only one type of API call he might have any. Also I don't think it's a good idea to run a for loop on a DB never, That something the worst I have ever seen design in my life. any then dev don't consider GC, they just code and blame framework or language. Without realizing trade-off, internal working etc.. I am tried of listing about such devs.

    @atishayjain1141@atishayjain11416 ай бұрын
  • Ah, fuck callbacks in Node.js. I'll just take the L in performance, the maintainability is well worth it. If I wanted big performance, I would just write it in Go.

    @NatoBoram@NatoBoram5 ай бұрын
  • i miss flash/java applets

    @fu886@fu8866 ай бұрын
  • Mercurious. People who are interested in merpeople but not fully sure about it yet

    @mattymerr701@mattymerr7016 ай бұрын
KZhead