Scott Benedict is the king of assessment, so if you have never before pondered accurate assessment, standards-based assessment, etc., you should probably hop on over to his site, Teach for June, and spend the next few weeks there before you come back and finish reading this post. If you have already reached the conclusion that it is essential to create and administer assessments that accurately demonstrate whether or not students have mastered whatever skill you are assessing, then you can keep reading :)
The Common Core has a bad rep (with good reason, I would say), but that doesn't mean that all of its content is bad! One of the things that I appreciate are the 10 Anchor Standards for Reading. I appreciate them because they have made it much easier for me to write excellent reading assessments because they have gathered together lots of different ways to comprehend and demonstrate comprehension of a text in one place! In order to truly determine whether or not your students have understood a text, it is essential to ask high-order thinking questions in addition to your basic fact regurgitation questions. If a student truly understands a text, whether fiction or non-fiction, that student should be able to answer questions that require them to summarize the text, apply what they read to a new context (for example, "Which of the following is an example [not given in the text] of the concept described?"), interpret selected portions of the text ("What is another way to say [excerpt]?"), evaluate portions of the text ("Which of the following statements did the author best support?"), and more. Writing these questions isn't difficult, per se, but it does require intentionality and forethought. The CCSS have done this for us, woohoo! I have a list of the standards posted near my computer, and I reference them whenever I write a reading assessment. If the assessment consists of five questions, I try to hit five different standards with my five questions. The end result is a challenging assessment; but that is because it is a quality assessment. When I first began to administer assessments with varied, high-order thinking questions to my students, their reading scores plummeted. Part of this was due to poor test-taking ability, but part of it was because my view of my students' reading comprehension was inaccurate due to my formerly poorly designed assessments. The greatest benefit of changing the way that I wrote assessments was that it forced me to be a better teacher (continuing instruction until my students had truly acquired whatever material we were studying), but a nice side benefit was that my students "strengthened their core". By this, I mean that they improved their test-taking ability effortlessly as I incorporating higher-thinking questions into my daily instruction through comprehension checks and discussion. (Side note: While I always administer reading assessments about Spanish passages in English (see Scott's website for the reasoning behind that), we discuss many of these questions in Spanish throughout each unit as we talk about stories and other input that we work with.) I don't really care about improving their test-taking ability other than it was a really great 'brag' for my Spanish program when it came time to defend my request to add an additional language teacher at the end of the school year!
I created this handy-dandy infograph for you to use when creating your own reading assessments, both formative and summative. Since the first two Anchor Standards for Reading contain multiple tasks, I broke them down a bit. Standards 7 and 9 are not included because they apply only when working with multiple texts, and Standard 10 is not included because it is too general to be helpful when writing an assessment. Enjoy :)