Mine did that too. I replaced the green o ring around the distributor, but it ended up still leaking. Bought a new distributor, still leaking. So I put a little bit of rtv sealant against the mating surface of the distributor and the head and it hasn't leaked since. Some actually put a bit of sealant on the inner portion where the distributor slides into. That works fine too.
If yours is leaking internally, you can rebuild it and replace the seal. I tried to do that with mine, but I couldn't get the damn pin out.