Beta is a stage in the release of software where it is functional, but possibly glitchy. There are two types of beta: Public Beta, and Closed Beta.
Public Beta: is where the entire software is open to the public. Anyone can use it, and anyone can test it without an invitation. It is often very close to being finished, but there are still glitches, and people report glitches more commonly than the final version.
Closed Beta: is when an invitation (or key) is required to join the beta. This is normally the first stage of beta.