Muwatta Malik, Book 45, Hadith 1609 (In-Book)

Malik related to me that he had heard that when Umar ibn Abd al- Aziz left Madina, he turned towards it and wept. Then he said, "O Muzahim! Do you fear that we might be among those that Madina casts off?"

وَحَدَّثَنِي مَالِكٌ، أَنَّهُ بَلَغَهُ أَنَّ عُمَرَ بْنَ عَبْدِ الْعَزِيزِ، حِينَ خَرَجَ مِنَ الْمَدِينَةِ الْتَفَتَ إِلَيْهَا فَبَكَى ثُمَّ قَالَ يَا مُزَاحِمُ أَتَخْشَى أَنْ نَكُونَ مِمَّنْ نَفَتِ الْمَدِينَةُ

sunnah.com

also referenced under: Book 45, Hadith 9 (USC-MSA)