Eliminating ripple voltage in biphasic square signal circuit without output capacitors

Hi all
I have a tense circuit which outputs a bi phasic square signal with some leakage current flowing to the gnd which fails the device under external lab tests.i figured this probably happens due to the resistors and capacitors connected to gnd at the output so I removed them.upon doing so, the signal is now having ripple voltages, which explains why the capacitors were there.is there zn alternative way to eliminate this ripple voltage without using capacitors to gnd? Perhaps a common mode choke at the output?
Would appreciate any idea.

What voltage is the thing working at, quite high if the voltage rating of the caps is anything to go by.
There is already a common mode choke in the circuit. If the circuit is there for the purposes of reducing EMI, then removing the caps will be an issue, and if the circuit is operating at a high voltage then discharge resistors may well be needed if there is any chance of coming into contact with it. So I guess a bit more information about the circuit would help,
cheers,
Richard