|
The NDERF Study
Methodology: A Closer Look |
NDERF Research Overview
We will now take a closer look at how NDERF research is conducted.
This is important, as extensive NDERF research findings are presented throughout
Evidence of the Afterlife: The Science of Near-Death Experiences. NDERF
research findings give a more in-depth look at many aspects of NDE than has been
previously possible due to the vast number of NDEs studied.
The NDERF website contains a survey with over 150 questions.
Responses from participants completing the survey are saved in the NDERF
archives. The survey data is placed in a database, and then merged into a word
document. The word document contains the survey questions and corresponding
responses submitted from the website survey. I personally review each
submission to determine if the account meets is a near-death experience, as
discussed in Evidence of the Afterlife: The Science of Near-Death Experiences.
Approximately half of all accounts submitted are determined to be NDEs.
A large number of prior randomized studies have found that responses
to Internet survey questions are as reliable as the traditional pencil-and-paper
methodology. This important point is discussed further in another section of
this part of the website.
The NDERF survey contains a question asking permission to post the
NDE on the NDERF website. Permission to post their account on the website was
given by virtually all. Of those giving their permission to post their
accounts, virtually all NDE accounts were actually posted. NDE accounts may not
be posted for ethical reasons if they strongly advocate illegal behavior, such
as drug abuse or suicide. Fortunately, this resulted in the exclusion of less
than ten accounts from posting. Approximately ' of 1% of all NDEs shared with
NDERF were excluded from posting on NDERF for ethical reasons. This percentage
of excluded experiences is so small that this would not be expected to alter
conclusions from the study of NDEs that are posted on NDERF.
All but a very few accounts were posted anonymously. The few
accounts not posted anonymously were special situations where the NDEr gave
explicit permission and there was a compelling reason to not post anonymously.
Examples of such situations include where the NDEr requested to share the title
of a book they wrote, requested a link to another website with more information
that identified them, or an emphatic request by the NDEr to not be posted
anonymously. Several NDErs gave approval for their account to be posted, then
withdrew permission later. In all such circumstances, the posted account was
promptly removed.
At the current time, many NDERF survey questions require check box
responses of Yes/No/Uncertain. Most questions in this format allow additional
narrative responses. Other questions allow only narrative responses. Still
other NDERF survey questions allow a range of responses. All survey responses
are placed in a database, allowing the responses to specific questions to be
researched accurately even with a large volume of data.
Responses of NDErs to a number of NDERF survey questions have been
researched in detail. A summary of research conducted in this manner will be
presented throughout Evidence of the Afterlife: The Science of Near-Death
Experiences.
In all prior NDE research, it has never been possible for interested
individuals and researchers to review the original source of material;
specifically the full NDE accounts. In Evidence of the Afterlife: The
Science of Near-Death Experiences, any interested reader may personally
review the original source of material. The response to each survey question
may then be considered in the context of the entire shared NDE account. In all
prior published NDE studies, if a quote from a NDE account was given, readers
could usually not review the context of the quote by reading the entire NDE
account. This is a very important advantage of the NDERF research methodology
and helps in the understanding of the quotes presented in Evidence of the
Afterlife: The Science of Near-Death Experiences.
This NDERF research methodology allows more accurate and unbiased
understanding of the quotes presented from NDErs. Most quotes presented in
Evidence of the Afterlife: The Science of Near-Death Experiences were edited
slightly. NDErs were given a copy of the edited quote to review prior to
publication. The edited quote was only published in the book if the NDEr agreed
that the edited quote was as accurate, or more accurate, than the unedited
version of the quote that the NDEr's originally shared with NDERF.
The NDERF research is based on studying the largest
number of NDEs ever reported by using a relatively strict definition of NDE and
using the NDE Scale. Studying large numbers of NDEs allows more confidence in
the study findings than would be possible from studying smaller numbers of NDEs.
No study of NDEs could possibly survey all NDErs everywhere.
Therefore, all research studies of NDErs are samples of a small portion
of the total population of NDErs. Consequently, there is some inherent
statistical uncertainty in any survey of NDErs. The large number of NDEs in the
studies presented throughout Evidence of the Afterlife: The Science of
Near-Death Experiences helps the confidence in the research findings of both
quantitative and non-quantitative content of NDEs.
NDERF Research- An Even Closer Look
More than 1300 NDErs have shared their experiences with the NDERF
website over the years. These NDErs completed the detailed NDERF website survey
in addition to sharing a narrative of their experience. This quantity of data
provides the opportunity to make an important contribution to the scholarly
understanding of NDE. The largest previously published NDE studies generally
involved analysis of less than several hundred NDEs. From basic scientific
principles, the NDERF study of large numbers of NDEs allows more confidence in
many research findings than has been possible from previously published smaller
NDE studies.
Vast numbers of NDErs have taken the time to share the most
important experiences of their lives with NDERF. NDErs may spend over an hour
completing the NDERF website survey. Researching these NDEs using the best
methods possible is an important way to respectfully honor the efforts of the
NDErs who, out of the goodness of their hearts, shared with NDERF. The NDErs
never receive compensation of any kind for their efforts in sharing their NDEs
with NDERF.
Over the years, the cost of maintaining the NDERF website ranged
between many hundreds to several thousand dollars per year. Managing the
day-to-day operation of the NDERF website requires hundreds of hours each year.
In spite of this tremendous requirement of finances and time, I never personally
had anything for sale on NDERF and never solicited contributions. The NDERF
website is dedicated to the accurate and comprehensive presentation of NDE.
Another major focus of the NDERF website has been to be a service to others.
An increasingly important role of NDERF is the scientific study of
NDEs. NDErs themselves are very interested in the scientific study of NDEs.
The NDERF survey asks 'What could a national organization with an interest in
near death experience (NDE) do that would be of most interest to you?' In
response to this question, NDErs may select one or more of fourteen different
responses. The response most commonly selected by NDErs was 'Scientific study
of NDE.'
The following is an in-depth discussion of the methods used in NDERF
research. Understanding the NDERF research methodology will help explain why I
am confident in the validity of the NDERF research results, and the remarkable
conclusions of the NDERF study.
Virtually all NDEs posted on NDERF are shared via the survey on the
NDERF website. A few of the posted NDEs are shared as narrative accounts via
e-mail. These accounts shared via e-mail did not involve completing the NDERF
website survey.
The NDERF website can be accessed world-wide, in every country of
the world. The NDERF website survey has been translated by volunteers into over
20 different non-English languages. This allows the survey to be available in a
familiar language to the great majority of people world-wide. While the NDERF
survey requires a computer, internet access, and the ability to find the NDERF
website, this methodology allows the NDERF website survey to reach those who
might not be easily reached by other survey methodologies.
All experiences posted on NDERF were shared voluntarily by the NDErs.
Exceedingly few NDE accounts posted on NDERF are associated with any commercial
interest of the NDEr, such as a book or for-profit website by the NDEr. All
commercial interests the NDEr may have that I am aware of are disclosed when the
NDE is posted on NDERF.
All prior surveys of NDErs required that the NDEr's be contacted in
some manner. Most prior surveys of NDErs had the limitation of surveying NDErs
from a narrow geographic location and/or limited to members of a particular
organization. Prior surveys of NDErs are generally valid, but this illustrates
the problems associated with trying to survey a representative sampling of all
NDErs everywhere.
The NDERF survey is accessible to people worldwide, and over a
period of many years, with the only restriction being the need for a computer
and internet access. Thus, the NDERF survey methodology may be reasonably
considered to allow as valid a sampling of NDErs everywhere as prior published
surveys of NDErs. Hundreds of other published NDE accounts are available (from
sources other than NDERF) in books, scientific articles, newspaper stories, and
magazine articles. Substantial numbers of NDEs have been presented on
television or radio. Review of these NDE accounts indicates their similarity to
NDEs posted on NDERF. Many of the NDErs who shared with NDERF have also been on
a variety of media presentations and/or have participated in other scientific
studies of NDE. This further supports the conclusion that the NDEs on NDERF are
reasonably representative of all NDEs everywhere, and consistent with previously
published NDEs.
When NDEs are shared via the NDERF survey, responses are saved in a
security protected part of the website. The responses are downloaded from the
website, usually weekly, and placed in a database. The NDErs' responses are
then merged into a document containing all the questions asked on the NDERF
survey.
The NDErs' responses to the NDERF website survey are archived
exactly as submitted. For NDEs posted on NDERF, the accounts are edited only to
correct obvious typographical errors, to remove specific information identifying
the NDEr or other individuals by name, and to remove disparaging comments about
specific individuals or institutions.
NDErs are usually e-mailed a notice that their account is posted on
the NDERF website, and a link to their account. This is done both as a
courtesy, and to give the NDEr a chance to review the accuracy of what is
posted. It is extremely rare that an NDEr requests any correction of their NDE
as posted on NDERF. This process helps assure that what NDErs share about their
NDE is presented on the NDERF website as accurately and comprehensively as
possible. While this process takes a tremendous amount of work, it is vital to
assure the reliability of the NDEs shared on NDERF.
Two versions of the NDERF survey have been used for research. The
original version of the survey was used from January 1999 until September 2004.
This original survey was a very detailed questionnaire, allowing 131 possible
responses to a variety of questions regarding NDEr demographics, NDE content,
and NDE aftereffects. A total of 413 NDErs meeting study criteria shared on
this original version of the survey.
For many questions, this original version of the survey allowed a
response of 'Yes', 'Uncertain', and 'No', and a default response of 'No
response'. Many NDE researchers believe a significant percentage of NDEs are
'ineffable'; meaning not able to be expressed in words. With the original NDERF
survey, there was concern that NDErs might be unwilling to complete the survey
if they did not believe a response of 'Yes', 'Uncertain', or 'No' was an
appropriate answer for the survey questions.
If NDErs were unwilling to complete the survey, the most important
part of the survey, a narrative of their NDE, might never be available. For the
413 NDEs analyzed from submissions to the original version of the NDERF survey,
the option of 'No response' was selected between 2.9% and 15.2% for questions in
this format, with an average of 7.0%. After reviewing hundreds of NDEs shared
on the original version of the NDERF survey, it became evident that NDErs were
generally comfortable expressing their experiences in words. This suggested the
option of 'No response' was not necessary.
Very few NDErs expressed concerned about the length of the original
version of the survey. Thus, the original survey was expanded. Although some
questions were eliminated, many new questions were added to help better
understand NDE.
In September 2004, the new version of the NDERF survey was
launched. The most important survey questions required a response from the NDEr.
This survey excluded the option of 'No response' for all questions. The new
survey allowed 150 possible responses to questions regarding NDEr demographics,
NDE content, and NDE aftereffects. A total of 613 NDErs meeting study criteria,
and with an NDE Scale score of seven or higher, shared on this new version of
the survey.
Another major change in the new version of the NDERF survey was the
addition of 16 questions composing the 'NDE Scale'. The NDE Scale is important
and widely used in scholarly NDE research. This scale was developed in the early
1980's by Dr. Bruce Greyson. The 16 questions in the NDE Scale ask about
particular NDE elements, and their depth. Reponses are scored from 0 to 32,
with a score of 7 or above necessary to qualify as a NDE. This allows NDE
researchers to be more consistent in comparing the results of surveys of
different groups of NDErs.
The NDE Scale
is not perfect. This is illustrated in the following example of an individual
crimially poisoned by strychnine (a highly toxic substance used in rat poison)
and later told by paramedics she experienced cardiac and respiratory arrest:
It slowly dawned on me that something was terribly wrong. I watched the ducks swimming in the water... that was the last thing I saw, before I died' I went up, flew upwards to the crowns of the trees' I could see what was happening on the bridge, even though I was not on it. I saw that my boyfriend was not there yet. (Which he later confirmed). Even though I was not on the bridge, I could see what was happening there. It was as if I could see anything I wanted to see. There were no limitations. I was still in the air' (https://www.nderf.org/Experiences/1dominique_s_nde.html, NDERF Post database #2657)
This experience had a NDE Scale score of 6, which would not qualify as a NDE. However, most people would accept this experience as a NDE.
Returning to the discussion of NDERF research methods, data from all
NDEs shared on the NDERF survey are placed in a database and a spreadsheet.
Spreadsheet algorithms have been developed to allow automatic and rapid
calculation of a variety of descriptive statistics as new data is added. This
is especially helpful as NDERF is accruing well over 200 new NDEs yearly, with
the rate of accrual increasing.
Several strategies were prospectively developed to determine the validity of
responses to the NDERF website survey. Both the original and new versions of
the NDERF survey were planned to contain redundant questions; questions asking
the same concept in a slightly different manner. The surveys were designed this
way to help assess the consistency of NDErs responses to the survey questions.
For example, if NDErs completing the survey were not accurately responding to
the questions, it would be expected to find poor internal consistency. Poor
internal consistency would be evident by significant variability in responses to
similar questions. On the other hand, if the respondents were accurately
completing the NDERF survey, responses to questions should have good internal
consistency. For the responses to redundant questions reviewed, there was
substantial consistency.
My background as a physician helps me determine if the
life-threatening event and subsequent course of medical events described in NDEs
are medically plausible. In the over 1300 NDEs shared with NDERF, a substantial
number of NDEs described medical events that were surprising or even
remarkable. However, I have yet to review a NDE account shared with NDERF where
the medical events described seemed so non-credible that I felt the account
should be considered invalid.
My medical background is important in confirming that experiences
considered to be NDEs actually occurred in association with a medically
life-threatening event. NDE researchers lacking a medical background may
encounter difficulty in making this critical assessment.
A major advantage of posting NDE accounts on NDERF is that this process essentially eliminates the possibility of posted NDEs being 'copycat' accounts. A 'copycat' account means all or part of the NDE is copied, or plagiarized, from another source. The NDERF website has well over 300,000 pages read by over 40,000 unique visitors to the website each month. If any posted experience is a 'copycat' account, it is highly probable it will be identified by one or more of the vast numbers of NDERF website visitors.
In the history of NDERF, only one copycat account was ever posted.
This occurred in the first year of NDERF's existence. This account was shared
via a personal interview, a process we no longer use, and was not shared via the
NDERF survey. After this account was posted, we were promptly notified by a
website reader that this account duplicated significant portions of a previously
published NDE. The copycat account was immediately removed. If, at any future
time, any NDE posted on NDERF is found to be a copycat account, it will be
excluded from NDERF research.
The enormous number of people reading NDEs posted on NDERF helps
assure their validity. This is a significant advantage unique to the website
survey research methodology conducted by NDERF. The NDERF website has an
associated public bulletin board, called the 'Forum'. The Forum began several
years after the NDERF website was started. The Forum is quite active, with well
over 100,000 pages read monthly. This provides another mechanism to present any
concerns about the validity of any posted account.
The NDERF survey itself is a strong disincentive to share falsified
accounts. There were 131 possible responses on the original NDERF survey, and
150 possible responses in the new NDERF survey. The new survey requires
responses to all key questions; thus it is not possible to skip many survey
questions. It probably takes at least 45 minutes, possibly much longer, to read
and respond to the survey questions and type in narrative responses. A
confabulated account would almost certainly not be as clear in the mind of the
contributor as the remembrance of a real NDE. It would be enormously difficult
to consistently respond to the detailed survey questions based on a confabulated
account.
Only two clearly 'joke' accounts have ever been submitted to NDERF. These accounts were submitted in the name of a well known adult movie actress, submitted at approximately the same time, and contained responses to survey questions that were clearly intended to be a 'joke.' This is a startlingly low percentage considering over 1300 bona fide NDEs have been shared with the NDERF survey over the years. The NDERF survey appears so complex and time consuming that 'joke' submissions are overwhelmingly discouraged.
NDEs are generally posted on NDERF anonymously as an ethical
necessity. Posting NDEs anonymously also serves as a substantial disincentive
to share a fraudulent account, as the contributor would never receive personal
recognition for their deceit. We have yet to post an account where we were
later contacted by the contributor to inform us the account was fraudulent.
Some NDErs who have never shared their experience with another
person completed the NDERF survey. A NDERF survey question asks 'Have you
shared this experience with others?' Of 613 NDErs, 90.4% responded 'Yes',1.1%
responded 'Uncertain', and 8.5% responded 'No.' The 8.5% responding 'No' to this
survey question is a small, but non-trivial percentage of NDErs. Reaching this
group of NDErs, who might otherwise be unwilling to share their NDE and
participate in NDE research, represents another advantage of the NDERF website
survey methodology.
Respondents to the NDERF survey must contribute the substantial effort of typing out their own responses to the survey questions. Thus, NDErs completing the NDERF survey may be more motivated to share their experience than NDErs surveyed via personal interviews, where less personal effort by the NDErs is required.
NDEs are complex experiences that might be difficult to express in words. The NDERF survey requires NDEs to be shared in written form with words. Fortunately, the NDErs completing the NDERF survey are generally confident with their ability to accurately convey their NDE. A NDERF survey question asks 'Did the questions asked and information you provided accurately and comprehensively describe your experience?' This question is intentionally worded to encourage a response other than 'Yes' if there is any aspect of their NDE that the NDEr believes is not accurately conveyed on the survey.
In the original version of the NDERF survey, 378 NDErs responded to the survey question. Of these, 312 (82.5%) responded 'Yes', 38 (10.1%) responded 'Uncertain', and only 28 (7.4%) responded 'No.' In the new version of the survey, response to the preceding question was required. Of 613 NDErs responding to the question, 518 (84.5%) responded 'Yes', 54 (8.8%) responded 'Uncertain', and only 41 (6.7%) responded 'No.' Responses to this question strongly suggest NDErs themselves are generally satisfied that the NDERF survey, and the answers they give, accurately and comprehensively conveys their experience.
This is by far the largest group of scientifically studied NDEs with
the use of the NDE Scale ever reported. As noted previously, larger numbers of
NDErs surveyed allow greater confidence in the survey results. This is
applicable to studying the non-quantitative narrative content of the NDEs. The
large group of NDEs in the NDERF study allows more confidence in the findings of
both quantitative and non-quantitative NDE content than has been possible in any
previous study.
It is unlikely that any other NDE research studies published in the
foreseeable future will be able to include as many NDEs as the NDERF study. It
is enormously difficult to access large numbers of NDErs who are willing to
participate in a study. The NDERF website, due to its enormous visibility on
the internet and excellent reputation, is uniquely positioned to encourage NDE
contributions for scientific study. There are approximately 48,000
links on the internet to the NDERF website. Even with NDERF's exceptional
visibility on the Internet, it took over 9 years to receive 1300 NDEs.
Previously published NDE research studies often relied on excerpted
quotes of NDErs as a major line of evidence for their conclusions. Publishing
excerpts from the NDE accounts was usually necessary because the full text of
the NDEs were usually too long to publish due to space constraints. In these
previously published NDE studies, there was generally no possibility for others
to review the context of quoted NDEs by reading the entire NDE. In the NDERF
study, readers have full access to the original source of data. This should
significantly increase confidence in the findings of the NDERF study. This is
especially important as many of the findings of this study are profound.