Can my employer force me into joining a provident fund?

It is completely legal. If you are eligible to join the provident fund in terms of the provident fund rules, then you are required by law to join the fund. Your employer is also not required by law to make a contribution; however the contribution should be structured as a salary sacrifice i.e. made in the name of the employer, so that your contribution is not taxed. The last point falls away in terms of new provident fund laws that come into effect on 1 March 2015, you will then be allowed to claim the contribution in your name as well. It is in your interest that you put money aside for your retirement, so you should be grateful that you now have a permanent job that also enables you to save for retirement.