I think that would be very tricky to decouple variables here. The vaccine seems to reduce mortality from COVID, but if the vaccine were killing x% itself, the vaccine would only have to be more effective at saving from COVID than killing for that safety signal to be lost. In short, comparing vaxxed vs unvaxxed mortality alone isn't likely to tell you if you have a vaccine safety signal.
You're right, you'd have to test whether people in each group are infected with COVID, and correct for that. Should still be doable. Would also need to correct for age and co-morbidities (doable), and also for differences in behaviour (hard).