You need to read the instructions that came with your LEDs closely.
Some LEDs are drop in and require the ballast be in place to operate.
Some require electronic ballasts others will work with the old magnetic ballasts or the newer electronic ballasts.
Some of the LED tubes are direct wired I have seen some that require the hot on one end and the neutral on the other end others require the hot and neutral on the same end and no connection on the other end.
Yes is you connect a direct wire LED to a ballast you can damage the LED and vise versa can be a problem.