What should we do with "*-choice" questions?

Question

Some of our questions can be categorized as some kind of *-choice question. The most common seems to be language-choice, but one could see the same type of questions asking about any standard teaching choice, for example IDE-choice, OS-choice, etc.

I don't believe that there's anything inherently wrong with these questions, but I think that answers to the questions can only be useful if the asker explains more about the context that they're making the choice in, for example by including information about the age group, CS background, language, or any other pieces of important context.

However, in some cases the asker omits this information, making it harder to write a complete answer and making it unlikly that the answer will actually help the asker. A good example is this: Is it ok to use logic programming(like Prolog) as students' first language? Recommended IDE for teaching web development to beginners. Even though it does include relevant context, the answers ignore that and are just peoples opinions on the "best" IDE.

What should be done about *-choice questions that fail to include relevant background information?

Answer 1

Well ..., for starters, without the limitations and context you mentioned, *-choice questions are just shopping list questions.

I am not a fan of closing such questions, just because they are *-choice types, and will inevitably get some POB answers. I do support the concept of trying to get them "focused" so that the OP actually has a good chance of receiving a useful answer, and to avoid the wasted effort of other users chasing the wrong thread with their attempts to provide an answer.

In the main, though not 100% of the time, I think such questions can be re-written to: i) give the relevant context and limitations the OP faces, ii) include the objectives, or problems to be solved, by the final choice, and iii) gather criterion and insights useful to the OP in making their own choice. Written that way the question aims to avoid answers that are basically a list of options with the author's opinion assigned to each one.

In addition, I think we ought to be on watch for answers, to this type of question, or any other, that are opinions. Field reports, what worked, didn't work, or was tried in some fashion, are not opinions - when presented as a report of:

We (I) did/tried <this>, and the results were <that>.
Here's what we (I) learned about <choice> from the experience.

Such a "report," presented in an answer about considerations and criteria for evaluating available options, can be very helpful to the OP. The "opinions" of random Internet users are likely not as helpful.

I do support the goal of turning *-choice questions into better questions, somehow. Obviously, placing the question [on hold] with that objective is the best way to reach that goal. There are, however, two things that we, as a community, need to have in place before that can be effective.

First, though apparent it must be stated, we need to have not only a consensus that it's the right choice, but a willingness, and commitment, to follow through and VTC such questions by those able to do so. Saying "I agree. Good idea." on this question doesn't mean much if the same users then turn around and answer such questions as presented rather than voting for them to be closed until fixed, and withholding an answer until they are remedied.

Secondly we need to have a "definitive" question, or answer, on Meta that guides the repair of such questions. Something that can be linked to in the comments on the question along with the VTC. The post should cover, in details that are not vague, what such a question should contain, and ask, that makes it acceptable. Pointers as to elements that improve beyond "acceptable" would also be worth having in that "canonical" post. Links to questions which are "exemplar" can also help the future OP repair their [on hold] *-choice question to get it reopened.

---

As a final note, the cited question about Prolog as a first language is not a good example of a bad *-choice question. It isn't even about choosing. If the title question is taken at face value, it is a 'Yes' or 'No' question. If the body of the post is considered, it becomes a "why is this?" type of question. The former is strictly opinion, even with "evidence" to support that opinion. The latter is primarily speculation. Unless someone has been on the panel that selected a first language, after including logic programming languages in the "running," none of us can say "why" more universities don't use logic programming as the first language. Our experience may suggest possibilities, but never answer the question.

Answer 2

As a service to both the asker (to protect them from getting locked-in to their broad question by broad answers) and the answerers (to protect them from putting effort into writing answers that will have little to no utility), I suggest that questions like these should be put on hold, but we should try our very best to reach out to the askers requesting the relevant background information and we should be editing it in and reopening when possible.

It serves nobody well to have a broad question with a broad answer. It's not going to help the asker, who in reality has a situation with clear constraints which restrict them from certain options and points them towards others. It can even be actively harmful, as an answerer can make assumptions about the askers circumstance and point them towards an ineffective solution.

Broad questions also will tend to encourage long answers, which require time, effort, and thought to write. That effort is likely to go to waste if it doesn't take into account the askers circumstances, which it can't, because the asker hasn't included their contraints.

That's why it's helpful to everyone to temporarily put these questions on hold as too broad. This leaves comments open so that we can attempt to help improve the question, and presents no reputation penalty because there's nothing wrong with not understanding our requirements.

Answer 3

Some of the questions also just ask for opinion and so are weak. The problem with putting them on hold immediately is that the asker might not actually be able to fix it up (unregistered...).

I think you are trying to control what the users do even though it is intended to be something of an unruly mob. My solution would be to require registration (and a bit of earned rep) before asking questions.

That solution has been repeatedly and summarily rejected, however. But we get questions from people with no real interest here and no sense whatever of the culture local or SE as a whole.