To set the AM/PM symbol you can either use the control panel regional setting. Depending on your version of Windows, this may be in slightly different locations.Normally you reach it through "Change date,time, or number formats" and then "Additional settings" and the "Time" tab.
You can also set this by using the following registry keys in HKEY_CURRENT_USER\Control Panel\International and set them to the desired symbol :