Sahih al-Bukhari, Volume 5, Book 58, Hadith 171 (USC-MSA) ⓘ

Narrated `Amr bin Dinar and 'Ubaidullah bin Abi Yazid: In the lifetime of the Prophet there was no wall around the Ka`ba and the people used to pray around the Ka`ba till `Umar became the Caliph and he built the wall around it. 'Ubaidullah further said, "Its wall was low, so Ibn Az-Zubair built it."

حَدَّثَنَا أَبُو النُّعْمَانِ، حَدَّثَنَا حَمَّادُ بْنُ زَيْدٍ، عَنْ عَمْرِو بْنِ دِينَارٍ، وَعُبَيْدِ اللَّهِ بْنِ أَبِي يَزِيدَ، قَالاَ لَمْ يَكُنْ عَلَى عَهْدِ النَّبِيِّ صلى الله عليه وسلم حَوْلَ الْبَيْتِ حَائِطٌ، كَانُوا يُصَلُّونَ حَوْلَ الْبَيْتِ، حَتَّى كَانَ عُمَرُ، فَبَنَى حَوْلَهُ حَائِطًا ـ قَالَ عُبَيْدُ اللَّهِ ـ جَدْرُهُ قَصِيرٌ، فَبَنَاهُ ابْنُ الزُّبَيْرِ‏.‏