Sometimes it can be easy to know a volcano is extinct. Take the ones in the UK for example (which are mainly in the Lake District and Western Scotland). These haven’t erupted for more than 60 million years, so we can be pretty sure that they’re extinct.
But you’re right, a lot of volcanoes are dormant. This means that they are not erupting at the moment, so there is no lava coming out. So how can we know that they will erupt again? One way is to look at the ages of the old lava flows and see if there is any cyclicity. So, for example, we see that Yellowstone in the USA erupted about 2.1 million years ago, about 1.3 million years ago and about 630,000 years ago. So it seems that the typical pattern at Yellowstone is to be dormant for about 700,000 years, and then to have an eruption (this is a very long dormant period, most are shorter than this). This would tell us that Yellowstone isn’t extinct, it’s just lying in wait for the next eruption.
Another way is to look at the behaviour of the volcano. For example, a volcano that is dormant may still experience small earthquakes underneath it as magma moves around at depth. There may also be geothermal activity – geysers, boiling mud pools, gas vents – that would tell us that, while the volcano is currently dormant, there is still plenty of hot magma down at depth, waiting for the right conditions for the next eruption.