sci·ence fic·tion
Noun
Fiction based on imagined future scientific or technological advances and major social or environmental changes,
Merriam-Webster
Like others have said in this thread, science fiction involves futuristic/scientifically advanced technology and its impact on humanity. Even movies that seem nothing more than alien monster stories generally involve a form of advanced technology to bring the elements of humans and aliens together in order to play out that story. For instance, the movie Alien is science fiction because not only does it have deep space travel (light speed or more) as an element (brings the humans to the point of meeting up with alien life, your "social" and "environmental change" right there) but it also has the implied element of genetic development of the alien life form (as a weapon), which would fall under the heading of scientifically advanced technology.
As for a more abstract answer to your query, all I can say is when a story (movie, TV show) demands that the viewers continually suspend their disbelief instead of giving rational, scientific explanations for the miraculous things that occur in the story, then it enters the realm of fantasy.
There can be elements of fantasy within a scifi setting, like the fact that in the ST universe the Federation recruitment requirements for women be that they all look like Barbie with big ta-ta's and have a high sex drive. But that's just marginal, nerdish, masturbatory fan boi stuff, it doesn't really change the scifi backbone of the story.