Are "accuracy" questions too broad?

Question

Recently we've had two questions about how accurate particular sources / tests are.

1. The set of questions Is this compiled genealogical record generally accurate? Stowell family of Higham, MA and the parent question How accurate do late 20th century Genealogical histories of New England in early 19th century tend to be?, which address the evaluation of large-scale compiled works, and
2. Is ancestry.com DNA test accurate with respect to native american descent?

These are all good questions to ask, but do they fit our format? Are they problems that can be solved, or are they too broad? How can we discover the problems that can be solved that lie within these larger questions?

In the case of number one, I tried to address the problem in a general answer that could serve as a canonical answer for how to evaluate information.

The second question is more tractable but still seems more a broad discussion question than a workable problem.

Your thoughts?

Answer 1

I think it is not so much the questions as the answers that tend to be too broad. Often it is easier to provide a general answer. However, it could be possible to provide a specific answer, if there is data about the accuracy of the resource. Here is an example a study of the accuracy of Wikipedia and encyclopedia historical articles. I don't know whether such a study exists for the resource in question, but the fact of the matter is it could.

Also, the use of the word accurate is quite different in these two questions. While it is used broadly in the first question to ask about the usefulness of a resource, the second question is more of a statistical question of accuracy. It is really a question about the sensitivity (false negative rate, Type II error) of DNA ethnicity estimates. Again, I think that it is possible to find the data to give this question a useful and specific answer.

The beauty of asking questions in this question-answer format, without discussion, is that if we can't provide an answer now, someone may be able to in the future.

Answer 2

I see Is ancestry.com DNA test accurate with respect to native american descent? as examining the evidence for what is more or less a single "fact", and so I think it is a good fit for focussed Q&A.

On the other hand I think Is genealogy compiled in 1922 for Stowell family of Higham, MA generally accurate? would need the evidence for a large number of "facts" to be examined, even if it is only a sample of all that are in that book, and so I think it is not a good fit for focussed Q&A.

I agree with @vervet that both questions may ultimately be answerable, but of the two it is the first that would seem to be more likely to attract an answer in its current state.

Consequently, I would prefer to see the second question morphed into one that just looks at the evidence for a single "fact", or multiple questions to examine the evidence for multiple "facts". Learning whether the findings support the first half dozen "facts" (chosen randomly or at least not all close together) that are examined will give the asker some idea of how much accuracy they are likely to find in the work as a whole.