Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Some cold water on the claims here: https://open.substack.com/pub/stoneztao/p/the-new-hyped-gene...

tl;dr: The way they're able to claim a massive speedup is by simulating much simpler scenarios than those being simulated by other systems.



blogpost author here.

This is not exactly the claim. I’m kind of certain they do “fair tests” where they test the same scenarios in different simulators. The problem however is that these tests are not indicative of anything useful. Why should one care if you get 430,000x real time speed simulating a robot arm not colliding or moving any objects with low sim accuracy? This number is fairly meaningless since if you do machine learning (specifically RL) to solve some task (manipulation, locomotion etc) that speed will drop significantly. Moreover in the past most simulators typically benchmark against previous versions of themselves and on scenarios with more collisions / somewhat more realistic situations (picking up a cube).

The original paper that started the trend of GPU parallelized sims is this one isaac gym: https://arxiv.org/pdf/2108.10470

They test on 3 environments, all with multiple collisions spanning locomotion and manipulation and get some nice speeds. These tests are far more realistic and grounded in reality than the test that made the 430K number.


<Surprised Pikachu face>

Who would have thought simpler physics are easier to regress against /s




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: