Imaam Muhammad Ibn Abdul-Wahhaab (rahimahullaah) said: ‘’They (i.e. the pre-Islamic people of ignorance) carried out worship by whistling and clapping their hands.’’
Shaikh Saaleh Al-Fawzaan (may Allaah preserve him) said:
From the affairs (i.e. practices) of the Pre-Islamic days of Ignorance that the Messenger of Allaah [sallal-laahu-alayhi-wasallam] opposed them (i.e. the idolaters) was that they carried out worship-seeking closeness to Allaah- through whistling and clapping. Allaah (The Most High) said:
وَمَا كَانَ صَلاَتُهُمْ عِندَ الْبَيْتِ إِلاَّ مُكَاء وَتَصْدِيَةً َ
“And their prayer at the House (Ka’bah) was nothing but whistling and clapping of hands [Surah Al-Anfaal:35]
That is: The polytheists did not seek closeness to Allaah at the noble kabah, except through whistling and clapping.
Al-Mukaa means whistling and Tasdeeyah means clapping with the two hands and palms. They used to do this at the House of Allaah and named it prayer- seeking nearness to Allaah (Glorified and Exalted Be He) by way of it. This is from the [affairs] the shayaateen among mankind and jinn made fair seeming to them. That is because worship cannot be (carried out) except through that which Allaah (Glorified and Exalted Be He) has legislated; it is Tawqeefiyyah (i.e. established by the texts of the Qur’aan and authentic Sunnah). A person should not bring about anything (i.e. acts of worship) from (him/herself) or take it from someone else, while it is not legislated by Allaah to be established as worship to Allaah and nor does it have any origin in the Islamic legislation. And from this the prohibition of these two traits -whistling and clapping- can be established, even if a person does not intend to perform worship through them. Because there is resemblance to the polytheists in that.
And (with regards) to clapping, the Prophet [Sallal-laahu-alayhi-wasallam] specifically permitted it for the women when there is a need, such as informing the Imaam (i.e. by clapping) if he makes a mistake in the prayer. And that is because of the fitnah-if men are present- that may take place from her voice. It is neither permissible for a man to imitate the Kuffaar nor a women by way of clapping.
And if it the case that clapping is not permissible for men out of necessity with regards to informing the Imaam when he makes a mistake in the prayer, rather he informs the Imaam by way of Tasbeeh [i.e. saying سبحان الله], then it not being allowed when there is no necessity is even a stronger reason. And in this there is rebuttal against those men who whistle in the gatherings in resemblance to the kuffaar. [Source: Sharh Masaa-il Al-Jaahiliyyah. Page:104-105. Slightly paraphrased]