Genre fiction and mainstream fiction

A question has been raised whether genre fiction exists as a category, and whether mainstream fiction merits more attention and admiration. My answer is complex: genre fiction must satisfy two standards, not one, but is given a “pass” if it merely satisfies the standards of its genre, and falls below the acceptable standard of mainstream storytelling. When done correctly, written with perfect integrity, the best of genre fiction cannot be judged based solely on mainstream standards, because the author is satisfying both standards at once, and the tale cannot be analyzed into the genre elements and the story-telling elements.  

Genre fiction does exist, and these day there is little left of the once-prevalent condescension of literary highbrows toward the popular literature. Genre fiction is any fiction that forms a category of particular readership taste. What consistutes the boundaries of the genre depends on what the reading public buys. Genre fiction is judged as good and bad not on its storytelling quality only, but also on how well or poorly the specific genre standards are met.

Each genre is a genre because its has standards and conventions unique to itself. 

When a reader is in the mood for a Western, he wants a cowboys-and-injuns story. Even if the good story on other grounds, such as good plot and character, a guy looking to read a Western will not be interested in it if it does not have the particular tropes and props of a Western, or if those props are handled badly.
If this is unclear, let me use an example. I sincerely doubt any fans of Westerns went to see Will Smith’s WILD WILD WEST, a movie starring a giant steampunk-style spider-machine: or if they saw it, they liked it as a comedy, for its comedic aspects, or as science fiction, for its science fictional aspects, or as an adventure. I cannot imagine anyone going, “Well, I just saw HIGH NOON and THE SEARCHERS, and I want to see something in the same mood and setting and atmosphere, so I think I’ll go watch WILD WILD WEST.”
Mainstream stories concentrate on story telling only, and have only a single yardstick to judge them by: is OLD MAN AND THE SEA a good tale or not? Does THE DUBLINERS move you? Does BILLY BUDD make a comment on the human condition, the nature of justice?  Genre stories have two yardsticks: first, is it a good story qua story, and second, it is good genre qua genre.
For example, a horror story that horrifies is a success as horror, even if it is a failure in terms of sheer story-telling, the story as it would be with the horror element stripped out. A romance story that does not have a satisfying boy-girl romance, either a happy romance or a tearjerking tragedy, no matter how good it is as a story, is a failure as a romance.
A science fiction story is a success as science fiction if it includes that familiar sense of wonder which comes of contemplating what science might tell us, what the future might bring. SKYLARK OF SPACE is a good science fiction story because it is pure SF wonder, without any pause for characterization, and only the sketchiest plot. Why were the Fenachrone setting out the conquer the universe anyway? No one knows and no one cares. We want to see Richard Seaton blow up the Fenacrhone space-battlewagon with his third-order zone of force, assuming he can escape from the fourth-dimensional evil of the hyperplane.
Naturally, authors who can satisfy both sets of criteria, both the basic story-telling standard and the particular standard of a Western, a Romance, a Detective Story, a Horror story, a Fantasy or Science Fiction story, deserves high praise. It is difficult to do both, maybe impossible for anyone other than a genius. But we should not be too fulsome in our praise, because what the literary reader looks for is not what we look for: we need more than he. Science fiction is harder to write than mainstream fiction.
Let me use an example to say why. THE SEVEN SAMURAI is a classic of the Samurai story genre. The heart of this story examines the honor of theSamurai class and the unity of the farmer class in Japan. But the story is good without the elements specific to Samurai stories: you could strip out the Samurai elements, and make the story into a western (THE MAGNIFICENT SEVEN) or a cheesy space opera (BATTLE BEYOND THE STARS), or, if you wanted to be really, really lame, a bad swords-and-spaceships anime. But none of these ripoffs have the magnificent power of the original story because all that is being copied is the surface story, not the thoughtful and sad heart of the work.
In other words, any Science Fiction story that would make a good story with the science fiction elements ripped out, is a bad science fiction story, because then the science fiction elements are shown to be not integral to the work.