Sometimes the letter 'O' has a pronunciation like 'O', but other times has a pronunciation like 'A'. How can I know the difference? The pronunciation becomes like the letter 'A' or there is some difference?
The simple rule is whether the syllable with o is stressed or not. In stressed syllables it's pronounced as O while in non-stressed syllables it's pronounced closer to A (although not necessarily the same as stressed A).
A good summary can be found here (in Russian).
It's worth noting that whether or not vowel reduction is actually used in the speech, depends on the dialect. For example, This English wikipedia article has some more info. Note that the pair of its Russian counterpart - оканье and аканье - contain better explanation than the English version.
This is probably the most famous phenomenon of Standard Russian, it's called "аканье" (akanye, 'ah-ing') and it's a case of vowel reduction. Tons of materials are avaliable on the issue, Wikipedia has lots, too.
In short, the letter 'o' in an unstressed syllable is pronounced as 'a' with phonetic realizations as [ɐ] in the syllable immediately preceding the stressed syllable, and as [ə] in other unstressed syllables. Stressed letters 'o' are always pronounced as [o]:
молоко́ - [məlɐ'ko]
You can well pronounce both [ɐ] and [ə] as a pure [a], it won't sound as a foreign accent.