Over 200,000 Servers in One Place! Visiting Hetzner in Falkenstein (Germany)
More Info about Hetzner:
derbauer.hetzner.com/en/image-211013/
---------------------------------------------------------
Support me on Patreon:
www.patreon.com/der8auer
---------------------------------------------------------
Save 10% on your iFixit purchase: DER8AUER10
eustore.ifixit.com/der8auer
Pro Tech Toolkit: bit.ly/2JOFD8f
---------------------------------------------------------
Find my products at Caseking:
Delid Die Mate 2: bit.ly/2Rhv4y7
Delid Die Mate X: bit.ly/2EYLwwG
Skylake-X Direct Die Frame: bit.ly/2GW6yyC
9th Gen OC Frame: bit.ly/2UVSubi
Debug-LED: bit.ly/2QVUEt0
Pretested CPUs: bit.ly/2Aqhf6y
---------------------------------------------------------
My Equipment:
USB- Microscope*: amzn.to/2Vi4dky
My Camera*: amzn.to/2BN4h2O
(*Affiliate Links: If you buy something, I will get a part of the profit)
---------------------------------------------------------
Music / Credits:
-
Outro:
Dylan Sitts feat. HDBeenDope - For The Record (Dylan Sitts Remix)
---------------------------------------------------------
Paid content in this video:
- /
Samples:
- /
"please excuse the mess" _everything looks pristine, dust-free, brightly lit, neatly sorted into labeled boxes, and every cable is managed_ You tell this place is run by Germans, lol
lmao ikr
if Hetzner engineer come visit my company data center here, they sure will get heart attack seeing all the dangling wires that we call spaghettis monster here. hah
Just don't check the basement...
Hetzner is useless company
german quality be like
Oh shit, one of those might be my server box!
Same!
It's like when you get a child sponsorship and they send you the picture of the child.
Sadly mine are in Hetzner's Finland datacenter. I used to have servers in Falkenstein. So one of those could be my old server.
Oh yeah, I seen it 3rd row 4th down haha
Lol, what a surprise. New "Antics" video coming....?
Hey Roman, really enjoying the content you're posting on the EN channel. This video was super interesting, thanks for taking the time to work with Hetzner and letting us have a look at their operation. So cool!
Thank you!
@@der8auer-en I was wondering why all of your videos were no longer in both German and English, I thought maybe you gave up on making English videos and then I had the thought "what if he made a separate channel?" and I checked your channel info. I usually only browse on TV so I must have missed the message that you had a separate English channel. Was there a video where you announced this? If not , might want to make it more visible. Love your videos!
Hey der8auer. Just wanted to drop by and thank you for all the great videos, and especially a special one as a fieldtrip to a datacenter. Really cool. Hoping for more like this. Greetings from Sweden.
I love all of the in house testing! What a cool company.
Thank you for this and taking the time to produce, VERY VERY interesting and well produced.
Always fascinating to see data centers that work on scale. Great video.
That is so cool. Awesome of Hetzner to give you such wide ranging access as well. Thank you for this :)
yeah kinda weird usually its off limits to any kind of filming
Hi Roman! I’m from Portugal and I have worked at Hetzner for 4 years in Falkenstein. Thanks for showing up the DC and all the found memories that I have from their and from the Hetzner team.
Ena pá, um tuga que trabalhou na Hetzner? Estamos a ficar muito conhecidos lá fora hehehe. Cumprimentos!
@D R They are doing hosting. Maybe you met a black sheep of their customers. If you have trouble with traffic from them contact their abuse department. They are very fast and professional and will sort it out. They don't like being abused either.
It's not about you calm down
@@mansart26 nobody talked to you go away from me
I've been a happy Hetzner customer for almost 10 years now. Have many many servers with them including some of the models shown in this video. I've had a few hard drives fail (as they always do) and Hetzner replaces them extremely quickly, I'm talking under 30 minutes. Really excellent service and my only desire is for them to open more datacenters in more countries. North America (Canada, USA etc) would be wonderful as-well as Asia. I love their service and just want more regions for redundancy.
I've had them for 7 months and were amazing, their choice of drives used also is good. (I got an 7K400 6TB when I asked, thank god not a seagate BS) Everything just worked, including when we asked for a hard reset THEY even did the reset, and ALSO "replaced server cpu fan as it was faulty", i never even knew it was, after they did it i saw pretty low temperatures too... really good support in general
ashburn coming soon broooo cyamonnnn
Its not really good company for any serious bussiness, sometimes they shutdown entire company account and wipe all servers and basically tell you to go f. yourself. I know they do this alot, because they have cheap servers and some shady people are using their service, but still, they shutdown alot of people for no reason really, and with no way to defend or get back their data. Not very good place to run your bussines, also their anti-ddos sucks ass unfortunately. But for testing their servers are alright, but I would never run there anything critical.
@@eth_saver "sometimes they shutdown entire company account and wipe all servers and basically tell you to go f. yourself" you mean summerhosts? basically the hosts which resell servers from companies like Hetzner, mind you Hetzner is been in the business for QUITE some time now, I don't think they will ever do such a thing like one day notice to full company death. The Anti-ddos and stuff is true, their network isn't as good as competitors like OVH. I dunno about servers randomly shutting down with no reason, it HAD to violate TOS and then I can understand if it did, if you are respecting all the rules and not doing illegal BS I dont think it'd just shut down out of the blue I mean, the serious business is also just how it all works, you can just see the price and understand, it's not meant to have the best quality, hetzner is more like "quantity" over "quality" situation. I mean who uses consumer cpus in servers kek. serious business stuff should be placed in AWS or so, that's where they truly exist and are SUPER reliable
@@eth_saver we rent our servers at hetzner for over 3,5 years now. No problems at all. Good support, great hardware, awsome pricing, almost no downtime (i mean like maybe 4s/ 3 months or something like that for maintenance). The part of DDoS protection is true, but you can build decent protection yourself if you have the knowledge for it. I just wish that they would built a datacenter in Canada.
Very nice tour, I bet a lot of the viewers have stuff hosted at Hetzner, including myself! I also like how their newest building is constructed using an IFA W50 - also some efficient use of old hardware :)
What an awesome facility. I love that they're using their heads for the cooling solutions instead of just spending a ton of money on, and wasting a lot of energy with, air conditioners -- even using the saw-tooth roof that some old metal foundries used to channel hot air up and outside via the angled ceiling to the high windows (I bet those high windows even face north or northeast, to avoid as much direct sunlight/heat as possible).
Good point, then they could put solar on the south facing roof.
@@ArthurEmbleton солнечные панели здорово нагреваются
Very nice, been hosting my servers there for quite a few years now so it was nice to see how their setup looks. This also explains why my nvme drives are hitting 90c on regular basis.
As satisfied customer of Hetzner it was really nice to see how they datacenters look like. I have been working as engineer/specialist for hosting company myself, so I have quite lot if information how datacenters are runned. I have been customer for Hetzner, for few years already, and I do have auctioned server and several cloud servers. Videos like this make those big hosting companies to feel more aproachable.
Awesome. Thank you for the tour, Roman.
Very nice video ! Thank you for this, I really liked the lead acid battery room small tour :)
Thanks for the most interesting tour. I am amazed by the size of this operation and the total amount of computing power contained here. I am from the old school of mainframe computers. First started working with computers back in 1965. One computer in an air conditioned room with a raised floor to circulate cold air. One CPU and 24K locations of memory in the typical system I first encountered.
Now I know i can sleep well since my servers are all there taken cared of like a baby. So great! Loved it.
Awesome video, really interesting to see how they manage all their servers!
i love hetzner, always used their services for hosting. nice to see how it works internally, thanks for the video! :)
Been a customer of Hetzner for years so really love this!! A++
15:30 Most times the DC from the batteries goes to the UPS. 3 phase UPS uses "double conversion" and what that means is that they take the AC from the mains and then convert it to DC (and that is where the batteries get connected) and then again the signal gets converted from DC to AC. This gives them a big benefit in terms of mains failure - 0 transfer time between mains and battery back-up, and the second benefit is the fact that this technology eliminates every "defect" in the mains - higher voltage, lower voltage, DC offset, swells... The DC bus on the latest gen of UPS devices is around 400-500VDC in order to achieve better efficiency (around 97% efficiency). Some new data centers uses DC power to directly power the servers. This is done in order to gain efficiency by eliminating the DC to AC invertor in the UPS and the AC to DC conversion in the server PSU. APC has some great White Papers on this topic as well as on cooling (as in warm countries it is impossible to use the outside air to cool the servers).
The batteries are just there to tide you over until the generators kick on. A few seconds.
Ah, nice! Thank you
@@1pcfred It depends. A lot of data centers have dual mains and they switch between them using automatic transfer switches that takes a few ms of time to switch (easy covered by the UPS). The diesel generator is a back-up of the back up most of the times. And the requirements for battery back-up is 5 minutes in order to be able to start the diesel generator and let it spin to the required rpm and it also needs to sync some times with the other diesel generators (usually they work in a N+1 redundancy configuration - if your data center needs 1MW, for example, you have 3*500kW diesel generators in order to be able to provide 1MW if one diesel generator fails for some reason). It takes around 1 minute but with diesel generators you never know.... they require a lot of maintenance and you never know what issue you can have with them.
I actually doubt they are using double conversion UPS'es in such a datacenter, because the efficiency is significantly lower than the line interactive/standby type. Normal good healthy SMPS power supplies have a capacitor bank big enough to handle the 5-20ms switching time of a standby UPS. As you are mentioning, many datacenters uses -48v DC systems as it is much much more efficient as you skip the conversion entirely. It would have been really nice if we were able to see how the batteries were wired, to get an idea of they are used. One thing they could be doing, (but i don't believe they are), is running ~230v DC directly to the power supplies, SMPS power supplies doesn't require AC, they run fine on DC, this way they would eliminate conversion AND be able to use standard PSUs. (only issue is that 230v DC is dangerous...)
@@HomelabExtreme another reason for 48V would be it is below the 'safe' limit of 50V (in most countries) so you dont need to treat it as 'live', use an electrician etc.
Awesome video -- Amazing place! Would love to see more "tour" videos of different types of tech facilities in the future!
Been a Hetzner customer for years, this was great, thank you!
Very cool tour. Awesome how they do all that in house testing and 3D printing. And lol at their “not perfect” cable management. Beats mine!
Videos and channels like these are doing great job at keeping KZhead to be useful and educational platform. Hats off to all participants of this video. Great Work! || Tolle Arbeit! Greetings from Slovakia! 🇸🇰 ☮ 🇩🇪
Amazing content! I was very curious about hetzner datacenter!
Amazing! Thanks for the tour 👍
11:51 I deeply relate to the lone memory chip that didn't make it into the cool chip club inside the funnel.
We have multiple virtual servers at Nurnberg but from the website Hetzner seemed like a small company, which it still is compared to AWS, but it's much bigger and profesional that I anticipated. I'm really impressed.
With these smaller companies, whats the point in buying their services compared to just running everything off AWS and Azure? My only experience with servers is running a minecraft server off google cloud tho. But im still confused how these smaller companies remain competitive yk?
@@honkhonk8009 Azure is beyond expensive, it can suck you dry out of your blood in no time. Hetzner is great, cloud machines are very powerful and cheap, ideal for small websites. We have dedicated hardware machine with Proxmox and it also works great. You have also unlimited internal network and with hardware machine even external, for the same price.
@@keenmate9719 AWS has all that and more.
Thanks for sharing - this is a refreshing look at a greener server farm.
FABULOUS tour, der8auer! You're a genius!!
My very first server ever was in the Hetzner datacenter. Nice to see the company is still doing well
Perfect video dude. I love these organisation of a data centre. I can remember the days I lived in Germany, how they care about thinks. In Germany, nothing is impossible, they didn't care what other say, they just made it right and precise. I'm missing those time really... Thank you too Hetzner for such of trust you give to this guy.
This is my new favorite video from Der8auer, thank you for this.
loved this video man, very well explained and interesting.
Hetzner - What an awesome company. I've been a loyal customer for many years. Thanks for bringing this great video to us.
That was an awesome tour!!!
Thanks for the intro, the compact servers with ATX motherboard that optimized for cooling is cool. And now I know why Hetzner's price is so much cheaper than other providers.
Fantastic tour of the data center, thank you.
Fantastic content, love to see servers from other countries and their different mentality!
Not much different, quoting Elon musk, "There is only one rule, Physics".
awesome! Just wish this was longer and even more detail, 60mins+ of going this stuff through would not get boring :)
As a long-time customer of Hetzner myself (and regardless of how old this video is as of writing) this has been a really exciting and interesting video. 👍
Completely enjoyed, that was very interesting! Thanks a lot! Subscribed!
Die Mühe die ihr da rein steckt, respekt!
Wenn nicht, denn...
Very cool to see somewhere like this. I can understand what they mean about trying to tidy things up with the HDD test racks because if you're using that kind of setup a lot (which they clearly are) then balancing hard drives and connecting cables like that is going to be frustrating (I worked on radar kit where we had a similar test setup and it was a real pain). A frame with a backplane would work better, a row of slots you can plug the drives into with LEDs at each HDD position.
This was a really interesting video. Thanks for making it!
Excellent hoster, never had a problem, support is always awesome and the adminpanel is great :-)
Ahh reminds me of the good old days, before everything went corporate. Makes me like them more seeing the normal and custom gear.
what do you mean "before everything went corporate"?
@@tddt2227 I think he meant before companies started using AWS and companies no longer directly managed the hardware of their servers.
@@Gamesational1: Guessing that's AWS.
Very interesting! Such a high level of hardware engineering! I would love to work in such a data center!
A very interesting tour and explanation, many thanks.
Super cool to see Hetzner insides... very good provider for years :)
OMG! I still use i7-920 too. It is still ok at gaming and gets the job done.
This is great. It's interesting that Hetzner went with Asus to build their boards, I keep thinking maybe other brands aren't so bad but in the end I always stick with them. Though Supermicro is cool if you want to build your own server, I recommend it!
Great video Roman, love this kind of content!
It is interesting to see where my old server was hosted, I never knew it was just a tower. Now I've moved onto their AX51 servers which if they're in that newer part of the building, they look very clean
I'm studying for my CCNA 200-301 and Linux + currently , I have a fascination with servers and hope to be working in a data-center like this some day...thanks for showing this
Definitely one of my favorite episodes. This has place has nerd written all over.
Oh hey, I can see my server from here! I've been using Hetzner for years and I'm super happy with them, so seeing this was really nice.
This was quite ineresting. I'm a Sys Admin for ~25 years and there is always something to learn. Amazing that they custom build everything. What I would give to have that ability in my Data Centers.
Building those super slim and barebone server cases in house is genius. Must be saving them a fortune.
Really cool, thanks for providing this video
Thank you for this video. You did a great job.
I would be interested in seeing a report on fans much like Backblaze does with drives. They have to have awesome data on these. Love the fact they use almost exclusively air cooling. So much to like about their approach to this.
Hetzner is awesome, I got a great deal on a box through their reverse auction. You can't get a box with specs that are even close in the states. Those guys rock! Their dashboard and load process is pretty slick too! These guys know what they're doing.
Well sounds like they're incorporating in the US, or having to charge VAT tax to US citizens now. While this doesn't mean the end of the world, in a price competitive industry, it definitely affects their edge they had.
@@ryanpaaz They just have to be careful not to be blown up like the pipeline or robbed with fines like VW when entering that US market...
This is the coolest server farm I have ever seen. I especially like the old school tower servers with desktop hardware! This is what I always wished to have what I was a teenager, haha. It's like a home lab on steroids! Love it.
Awesome work! Thanks for the video!
I've been a Hetzner customer since 2014, love the company, prices etc. I currently have an AX-51-NVMe with a 10Gbps connection :)
why not use AWS? what was the determining factor?
This is awesome!
What an awesome setup. The layout of some of these Data Centers always amazes me with how uniform everything is. GO GERMANY!🤟
So nice to actually see the datacenter, where I have rented a server for so many years :D
Say he is well known for breaking things. Linus laughs at this.
It could be that I accidentially killed my 12900K today by dropping a screw on the mainboard. So much about that
@@der8auer-en seen worse humblebrags in my life
@@der8auer-en lol no day 1 review for you
He is the European version of Linus 🤣
@@der8auer-en That sucks, those are expensive too.
The main reason you wouldnt run 12V out to your servers is cable losses. You could, though, easily deign your PSUs to take AC or DC (most switchmodes immediately rectify it after all) saving you those inverter stage losses. Thanks heaps for the amazing insight (and to Hetzner too)
Yup, that's why it's usually 48V systems in data centers and then converted into 24V or 12V in racks. However I don't know if it would be 48V directly from batteries.
@@insoYT That makes a hell of a lot of sense! why 48V? the legal safe voltage for exposed installations is 50V at least in the US, you dont have to treat it as 'live'
Your analog phone run on 50 Vdc
@@user-jn9dl9px6r what's that?
Depends, if you were to design a system where batteries were separated "per server" in theory you could run everything off 12v but it would be an absolute mess and a lot of unnecessary thick cables.
nice walkthrough Roman!
didnt know you had this English spinoff channel, I use hetzner so appreciated the video, thank you.
9:18 "I would like to touch things, but I tend to break things, that's why I'm not going to do it" Immediately takes hold of a cooler
it's pretty hard to break a heatsink eh?
was less worried than touching a Threadripper again
@@mscd9676 I did drop my heatsink and bent the hell out of the fins, so...
With the standing HDD's being "unorganized" It looks nice and straight, but really easy to fall over, shocks you don't want to give HDD's while running.
Great video and presented really well too
Very interesting video. Thanks to you and Hetzner.
Would there be potential for solar panels on these roofs just thinking about the angle.
i was just thinking that, why they hadn't covered the roofs in solar panels already!
Well, he said they are already using green energy. I wonder if solar just isn't enough, or consistent enough (night time) vs. wind or hydro. It would be interesting to know why they aren't using some solar, though. I'd guess that it's too cloudy on average, that the investment into solar at that location just wouldn't pay off.
@@47CryXMA Maybe too much resources needed for solar panels? I guess that the consistency is also a big factor, the power output is changing with the second and that is bad for the power net in general. If they use thin film panels they could partly fix the consistency. Those panels start generating energy at very low light but the max power is also low.
Maybe they angled the roof away from the sun to keep things cooler? Edit: Actually based on google maps it looks like that's not what they were going for.
I personally think it'd be a waste of money in Germany. It's cold around 3 seasons of the year.
In Seinäjoki Finland we have datacenter wich heats city with district heat coming from servers
Nice to see this, i had many dedicated servers there :)
Ich habe gerade das deutsche Video gesehen und einfach großen Respekt dafür, dass Du das ganze parallel auf Englisch gemacht hast anstatt subtitles oder eine kürzere Version mit voiceover.
What is the model name of those blue IP KVM devices that are daisy chained on the back of the midi towers?
Why are those southern facing roof surfaces not covered in Solar Panels?!
Because solar panels don't generate enough electricity to offset the cost of purchasing/installing/maintaining solar systems.
This is so cool to see inside hetzner datacenters. Ive had few servers from hetzner mainly in Helsinki since i live there lol and i currently have a storage box there.
Very interesting. Thank you!
Contest, derBauer vs. Linus on who can drop and break the most.
Nobody is wiser than engineer. Except German engineer...
Amazing and thank you Falkenstein for letting him in (i'd have laughed him away)
Was very interesting, thank you.
Man, building servers there would be my DREAM JOB! Do you know if they are recruiting? Even better would be them expanding into Ireland so I wouldn't have to move!
It would get old
@@mrmotofy Not likely as hardware keeps evolving ;) First PC I've built I was 11 and trust me when I say that this has never gotten old for myself, quite the opposite, new hardware and/or technologies coming to market keep things interesting! Like when FLASH finally came of age on SSDs instead of USB sticks. There's always something new or improved around the corner and sometimes some deeply disruptive changes like the switch from AT to ATX that resulted in a considerable upgrade cycle although this may not be the best example.
@@wskinnyodden Yea...but building these servers is gonna be very repetitive as they seem to be virtually the same. So some might have a 100Gb Nic instead of 10Gb...big deal yes I remember AT vs ATX, I picked an ATX as it seemed the way it was going.
@@mrmotofy Well, that is just a challenge on how to get as many as possible going at the same time efficiently and correctly. Wouldn't be a first, I still recall the first time I had 10 systems on the work desk at once being installed and set up each for a different customer. (Christmas time for a computer shop back then)
Trabalhei no data center da Embratel nas olimpíadas do RJ irmão, foi meu último job depois que me mudei pra EUA, queria muito voltar pra área também seria um sonho tá num lugar desse de novo
as a dutchman i got a question. ever been to the amsterdam data center? supposedly the biggest data (or busiest data center in europe some even say in the world). i have seen videos on it and read alot about it. and its very impressive. most game servers you use in germany are hosted on that data center. Blizzard and steam for example are located in amsterdam. its pretty interesting. anyway nice video very informative.
@@El_Marzocco we just got a new huge google data center in the north of the Netherlands in Groningen. Microsoft just build another one in the netherlands in Middenmeer its in the province of north holland. well they got green light to build it its not their yet but it will come. its in the province of north holland. But the reason why they do this is because we are a tax evasion country one of the biggest in the world. so its cheap for big companies to build their business here. they dont have to pay taxes and what not. something they had to do in any other country in europe. so easy choice to build it here. also we have the best fiberoptic infrastructure in the world. maybe that got something to do with it as well.
Awesome video. Thanks!
Yes enjoyed very much. Thanks
You really need to talk about this EN channel more on the main channel, I don't even know until I speculate that you won't upload no English videos and I found this in the channels tab.
I think I was talking about it in the last 3 or 4 english speaking videos and then just started linking below the German video. Hope the word eventually just spreads
Wow. I though datacenters or any kind of enterprises are running only xeons and epycs. I always wondered why intel, amd or nvidia artificially lock up their consumer hardware. I though that no one is willing to run consumer hardware for enterprise use so it seemed stupid to me that they bother to lock up certain enterprise features. This changed my perspective. And AMD or Intel must be quite mad seing 3900x being used instead of epycs which cost twice as much :D same for i7 instead of overpriced xeons. Very very interesting video.
Hetzner isn't "enterprise grade", it's inexpensive hardware at an inexpensive monthly charge that doesn't make you sign a long-term contract.
@@PablumMcDump Thanks for the correction. I went to their website a see now that I can rent specific servers with consumer HW, I thought that they are using the ryzens and i9 for cloud. Still though I didn't know this existed.
I've been using hetzner for 3 years, and I love it. keep up Hetzner
Appreciate your tour. I am going to build my own server with mini tower PC😂
Id love to work here