You're right that it seems close enough, but only as long as we're talking about the time it takes a single vehicle to "fill up". Taking 2 minutes vs 8 minutes to fill up your car doesn't matter to you personally, but it is a significant difference to those installing and operating the fill up infrastructure since it takes 4 times as many charging points as it does gas pumps to serve the same volume of customers in a given period of time.

In a lot of cases that probably won't matter since chargers can be installed in more places than gas pumps and for gas stations that serve mainly local customers I'd expect demand for an equivalent charging station to be lower since some people will charge at home at least some of the time. But things like highway rest stops could be more of a challenge since you'd expect customer volume for EV charging to be similar to the demand for fuel so you'll need more charging stations at each stop to handle the increased time it takes each customer.