I don't believe I would completely abandon the teacher directed lecture in front of a class. I would however reserve the class lecture material for more engaging discussions where an forum of students could further an idea in a group setting. Key problems that are universally challenging to all students could also be taken up in a more formalized interactive lecture style.
One downside that I see is that students must wait to ask questions for material presented in the video lecture portion of the course. The ability to ask a question and immediately receive expert clarification could make comprehension difficult or lead to false understandings of the material. Another potential downside is that the amount of video production time that the teacher takes to produce the videos could be prohibitive. What is the best way to get around this time constraint for the teacher? Also is it necessary to flip our classroom when Khan Academy is so widely available.
Khan Academy TED talk:
https://www.youtube.com/watch?v=nTFEUsudhfs