Multi-server authentication makes convenient to benefit from services of various service providers on the basis of one-time registration through a trusted third party. Since, the users are reluctant to register themselves separately from all servers due to the hassle of remembering many passwords and other cost constraints. The multi-server authentication enables the immediate provision of services by the real-time verification of users on an insecure channel. The literature for multi-server oriented authenticated key agreement could be traced back to Li et al. and Lee et al., in 2000. Since then, numerous multi-server authentication techniques have been put forth. Nonetheless, the research academia looks for more secure and efficient authentication protocols. Recently, Chen and Lee’s scheme presented a two-factor multi-server key agreement protocol, which is found to be prone to impersonation, stolen smart card, key-compromise impersonation attack, and trace attacks. Besides, the scheme is also found to have the inefficient password modification procedure. We propose an improved protocol that counters the above limitations in almost an equivalent computation cost. Moreover, our protocol is supplemented with formal security analysis using BAN logic along with performance analysis and evaluation.