We had with 5.4.0 :). It is just the effort rather than complex. That means you should modify wherever the SQL queries to support MSSQL query everywhere in the vtiger code.
Why do you need an MSSQL? If it is only for the purpose of integrating other tools such as Reporting and big data, I recommend to use MySQL to MSSQL Replication. Where you can keep vtiger database as MySQL, but replicate its data to a MSSQL databasse as mentioned below
Write triggers in MySQL that capture insert, update, and delete statements in a log table (same as binlog)
Poll/Read the log table from MSSQL using ODBC and execute them, then delete those log entries.