After the Siege of Jerusalem in 637 Patriarch Sophronius agreed to surrender only to the Caliph Omar himself. Caliph Omar travelled to Jerusalem and accepted the surrender. He then visited the Church of the Resurrection (today better known as the Church of the Holy Sepulchre) where Sophronius invited him to pray inside the church, but Omar declined and instead he prayed outside, on the steps east of the church.
The current Mosque of Omar was built by the Ayyubid Sultan Al-Afdal ibn Salah ad-Din in 1193 to commemorate the prayer of the caliph Omar.
The current mosque is located at a different site than the one where Omar is believed to have prayed and where the earlier mosque was located, since it stands to the south of the church rather than to the east of it. This new position is likely due to the fact that the entrance to the Church of the Holy Sepulchre had by then moved from the east to the south of the church, as a result of repeated destructive events that affected the Holy Sepulchre during the 11th and 12th centuries.