Most people assume you need to push or pull on the ends of the tool bits to remove them, but this approach is awkward at best. It's far simpler to push them from the middle, as shown here:
1) Place your finger on the lip of the tool bit (shown here in yellow), just as it transitions from the hexagonal shank to the driver.
2) Push downwards, until the driver is sticking out from the bottom.
3) Pinch screwdriver bit and remove.