Dealing with the inherent subjectivity of CS Education

Question

This recent question made me (once again) think about how we should reconcile Stack Exchange norms and our (inherently subjective) content. We also need to remember that our goal is to help the asker solve their issue, and to provide a resource for future educators that face the same issue.

The problem

We cannot help the asker if they do not provide appropriate context for their question. So much of teaching is depending on things like subject matter, age group, teaching style, etc. Questions that do not describe the context of the question are unlikely to receive answers that are helpful to the asker.

In addition, without appropriate context, a question will receive a broad range of equally "correct" answers. With multiple answers that are equally correct, people will end up voting on the answers that are their favorites, rather than based on correctness. This phenomenon beaks the voting system -- a system which is designed to help future readers see the best answers first.

So, a question without appropriate context is neither useful to the asker nor useful to future readers. Without context, the asker cannot get a context specific answer, and a future reader will not see the best answer first. In addition, users are then encouraged to share their personal opinion, without backing it up with facts or even experience.

The solution

I recommend that we establish a rule regarding required context for questions. Questions lacking appropriate context should be closed until the required context is edited in.

What are your thoughts on this? What constitutes appropriate context? Is closing too extreme? Can you suggest a more specific rule?

Answer 1

There's already a close reason specifically for this reason – unclear what you're asking:

Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.

If context is necessary to be able to answer the question, but hasn't been provided, the question can be legitimately closed as unclear, to give time for the author to clarify.

Note that I said 'if' – there are plenty of questions where context is not particularly relevant, and demanding it where it isn't necessary might do more harm than good. A blanket ban on questions lacking context seems unnecessary, given that we already require sufficient information to answer.

---

A counter-intuitive observation about Stack Exchange – one that I feel has perhaps been eroded a little now compared to in the past – is that we're not here to help the people asking questions. Now, bear with me for a second here. I'm not suggesting we're not going to help askers, but the primary objective is – and always has been – to create a curated library of questions and answers which help a broad audience. The fact that the author gets help is largely incidendal to this goal.

Unlike your regular support forum, we're building something that can last beyond that one person's concern, and overly specific context can get in the way.

As a selfish, greedy person, when I search on Google, I don't really want to know how someone else solved their specific problem – I want to know how to solve my problem. An overly specific answer to one person's problem is not as useful, though perhaps easier to answer. I've participated on sites where there isn't a history of canonical questions that help a broad audience, and, frankly, it becomes tedious answering very similar questions, over and over.

So, perhaps I've now made you think that we should throw out all of our rules and guidelines, and encourage very broad questions. Of course, that's unworkable too, and there's probably a happy medium between the two extremes of too localised and too broad.¹ Exactly where this lies... I don't know. 'List' questions have been tolerated a little more than other sites already, but I don't think I can give you an objective rule for when a question becomes sufficiently specific to be okay. Sure, I can probably tell you what I think when I see it, but I think you have to trust the community's judgement to an extent, until there are serious problems coming from these questions.

That said, your specific question doesn't sit perfectly with me... I wonder if it invites more opinion than expert judgement, without a concrete example or two at hand. Answers explaining how you could judge plagiarism vs collaboration both in the specific example and more generally would then be very useful for future visitors, too.

---

¹Long ago, too localised was in fact a reason for closure – a reminder that we're not really here to help the one, but the many... then again, even longer ago, noise or pointless was a reason for closure, so not all past reasons were a good idea. I'm torn on whether overly specific questions are actually healthy for Stack Exchange, but that's a debate for another day.

Answer 2

As you suggest, education depends fundamentally on context, with more variables than you list. That doesn't make it "subjective" but it does make it subtle. The original idea of stack exchange (I suspect) was to have people ask questions about formally defined things (programming languages for example). But education is about people and their growth. It is a fundamentally different kind of thing.

Inexperienced teachers don't themselves have a lot of context to draw on. They haven't been around long enough to have seen a sufficient range of issues and certainly not long enough to have seen all possible solutions. Experienced teachers may be better at this, but only if they have consciously thought about their teaching, and many do not, seeing teaching as only a necessary, if painful, task they must do so they can get back to research.

However, a question with little context is still valuable, as long as the one who provides an answer has some context him/herself. That is because solutions in education aren't binary: do precisely this with no modification or embellishment. If I, for example, give an answer I expect that a future reader, having the question and answer in hand, but knowing their own context will be able to apply the "solution" in a flexible and non-trivial way. It might not exactly match what "I" said, but if it gives them a hint about how to move forward in "their" context then I have done some good. That is about as much as we can hope for when we can't work together with the future reader.

Therefore, I think your suggestion is misplaced. If a question provides so much "context" as to make only a single answer "correct" then that question and answer are essentially worthless to another future reader whose context will almost certainly not be the same.

Education, in my view is a personal relationship between a teacher and a learner. A formal relationship (professor, student) isn't necessary, but those who do it as a profession need to understand that and, given the constraints make that as real as possible.

Answer 3

Head-first and Once more unto the breach, dear friends, once more¹

I find myself straddling several points here. As a good and faithful servant of SE, I looked to Meta Stack Exchange for some guidance. Well, for a bit anyway. My largest concern was the part about the goal of getting answers for one, or for many, users. Partially from the OP's question and partly from an answer. I believe that the focus on that point leads to implicit guidance on many other points.

In several posts, across multiple sites and within the chatrooms, I've encountered the idea that the goal of SE, in general, is to be a repository of knowledge, curated by the users, for all of the Web to access. To quote from our site's tour page, "With your help, we're working together to build a library of detailed answers to every question about Computer Science education." Now, with only 8+ months in Beta, it's hard to imagine that we can even begin to answer every question about Computer Science education, but that is the goal, nonetheless.

Frequently I have seen it said that an answer that only works for one user, has no value to anyone else. The so-called "too localized" questions. I just knew there had to be a canonical answer on Meta SE about such questions. To my surprise I found the opposite. I've never heard of user Jaydles before, and I'm not 100% certain that the profile can be trusted, but it lays claim to "VP & GM of Stack Overflow." Maybe not a perfect spokesperson for the question, but reasonably close. For full-disclosure, there is a bit of debate between him and Jeff Atwood, who seems to be a rather authoritative voice on SE as well. In any case, the upshot of Jaydles answer is that a question that ends up helping tens of thousands of people is good, "[b]ut helping 100 people is pretty good, too, and helping 1 is better than 0". He adds, in large bold type, "Questions should NOT be closed simply because they only help one, or too few people, but patterns of questions that threaten the greater good should eventually be made off-limits."

Of course, Stack Overflow has been around a lot longer than we have, and there is no reason we cannot learn from their lessons, and avoid some problems they have figured out how to avoid. We also are not Stack Overflow, and have a different user expectation, and some of what didn't work there might work here, and some of what did work there might not do so well here. We will have to find, and strike, our own balance between helping users and building the site. Any good question is likely to help more than one person anyway. As the user base grows, currently only 4,089 users with 152 at 200+ rep, we might have to further limit questions using some criteria or another. I doubt that required context is it, however.

Considering that our stated objective is to answer every question about Computer Science education, we must allow every question. We cannot have the answer to a question not asked. Even with that in mind, we still should have some kind of limits. We cannot, for example, have the answers to questions such as "What is the best programming language to present to Masters Candidates in the Class of 2024?" Prognosticators we are not. Other questions, most notably lists, AKA "shopping questions", just don't fit well on most SE sites. (We are slightly more tolerant than some sites, and reviewing this post might be helpful in that regard.) Still, that leaves a lot of questions that just cannot be answered with good answers and that are not worth keeping around for the ages. Finding, and closing, those questions, after we figure out what they are, will be a "process" the site follows as long as it exists. What merits closure is likely to continuously evolve as our site marches through time as well.

The sample question given by the OP is probably not a good choice for presenting the OP's case either. It seems to be thought out, researched, and coherent. It provides enough context to present the problem succinctly without getting into the gray area of being a one-person answer. The subject does not need context relative to "things like subject matter, age group, teaching style, etc." In fact, the potential issue is relevant to all CS education as given, even though presented in a programming context. As such answers can be helpful to the asker, and to future readers.

Additionally, a question might receive answers from different contexts, as happens most of the time on other sites network-wide. All those questions may very well be equally "correct" within their context, or as a whole. As an example, this question on Stack Overflow received 4 answers in 13 minutes. According to the voting, all are "correct" and remain in the top 5 after the 7+ years the question has been there. That OP selected one as his choice of "best" answer, not the same as "the single correct" answer. From personal experience I can say that the top rated, and/or accepted answer is not always the one that solves my problem. Three examples where my vote went to a "lessor" answer recently have dealt with bin in the PATH environment variable, working with GPG keys, and installing the bootloader. The popup "hints" for the voting do not say "This answer is correct" and "This answer is incorrect", they say "This answer is useful" and "This answer is not useful." So far having such a collection of correct answers, with various degrees of overall usefulness to other users has not broken the voting system of SO, and I doubt it will break the voting of CSE either.

In summary, I oppose the proposed rule to require questions to include explicit context generally. A question that needs it can have comments added, or be put on hold, but only if the question needs it, the same as any other defect in the question would call for a hold.

---

The summary

What are my thoughts on this?

See above.

What constitutes appropriate context?

As much as is needed, and only as much as is needed, to make the question clear. Maybe a lot, maybe none at all.

Is closing too extreme?

Since SE calls it "On Hold" for the first five days, it is not too extreme when it is warranted. It is too extreme if the question can stand, as is, even if more context might be helpful.

Can I suggest a more specific rule?

Yes. A rule of "non-thumb." A rule of thumb seldom works in the majority of cases for the long term. Such rules help make spontaneous decisions when limited facts are available and a decision must be made soon. Otherwise the thumb accounts for only 20% of the digits on your hand (typically) and has an average success rate nearly the same. (For blocking the sun when you need the greatest field of vision, however, the rule of thumb is perfect.)

Basically, every question deserves to be judged on its merits, not on the merits of previous questions, even when "similar."

---

1) William Shakespeare, Henry V, Act III, Scene I