It just took an update of the DejaVu font on tile to have the local Georgian names rendered (thanks jburgess):
osm.org/?lat=41.539&lon=43.896&zoom=9&layers=B0FT

I was looking at the map sources on Wikipedia for მცხეთა/Mtskheta (http://tools.wikimedia.de/~magnus/geo/geohack.php?pagename=Mtskheta¶ms=41_51_N_44_43_E_type:city), and it seems OpenStreetMap is the only online map which shows Georgia in local script. Quite cool.