User blog comment:IamJakuhoRaikoben/Chapter 439: Albareth Empire, Review./@comment-110.54.171.194-20150608090406/@comment-105.158.135.177-20150608100146

Or he can only get his memories back when Makarov is around him, and loses them again when he leaves. Like Doranbo... i mean Mest said, Makarov has the keys to restore his memories.