Research Designs on speechBITE
- What types of studies are included on speechBITE?
- Where does speechBITE find the studies?
- How does speechBITE decide which studies go on the website?
- What is a clinical practice guideline?
- How does speechBITE decide which clinical practice guidelines go on the website?
- In what order does speechBITE list studies?
- How are studies rated?
- Which study designs are rated?
- Which ratings scales are used?
- What do I do if I don’t agree with a rating?
- Are the ratings on speechBITE reliable?
- Can I become a rater for speechBITE?
How do I…
- Access full text for articles on speechBITE?
- Report an indexing or rating error?
- Suggest publications for inclusion on speechBITE?
- Get help with searching speechBITE?
- Sponsor speechBITE?
Research Designs on speechBITE
What types of studies are included on speechBITE?
The following list provides definitions of the research designs included on speechBITE. These definitions are drawn and adapted from those used on PsycBITE.
A systematic review is an overview of primary studies that contains an explicit statement of objectives, materials and has been conducted according to methodology that is clearly defined and able to be reproduced. The aim of the review is to address a specific clinical question by assembling, synthesising and appraising the results of all relevant studies in the area, using methods which limit bias. A meta-analysis is one form of systematic review. This is a mathematical synthesis of the results of two or more primary studies that addressed the same hypothesis in the same way.
Randomised Controlled Trial (RCT)
An RCT compares at least two treatments/interventions (one of which can be a no-treatment group or wait-list controls), which involves random allocation. Participants are randomly allocated to groups for either the intervention being studied or the control/placebo group using a random mechanism (such as coin toss, random number table, or computer generated random numbers) and the outcomes are compared.
Non-Randomised Controlled Trial (Non-RCT)
This type of study is similar to an RCT in that it compares at least two treatments/interventions however the participants are not randomly allocated to groups. Outcomes of the treatment group and control group may be examined prospectively or retrospectively. The allocation of participants is carried out using a selection process (e.g. matching of participants according to sex, age or other criteria) or allocation by date of birth, hospital record number or alternation (this is referred to as pseudo-randomisation).
Case Series (CS)
A group/s or series of participants are exposed to one treatment/intervention where outcomes are measured in participants before and after exposure to the intervention. No control group is included. Detailed information regarding individual participants is often presented.
This type of study compares the effectiveness of interventions using the same (single) participant as his/her own control by assessing treatment effects over several different points in time as treatments are systematically applied and sometimes withdrawn. More information on single-case designs can be found under “Rating research quality”.
Where does speechBITE find the studies?
The references on speechBITE were identified through a comprehensive search of the relevant literature on eight databases. These databases include MEDLINE, Embase, CINAHL, PsycINFO, ERIC, AMED, LLBA, the EBM Reviews (incorporating references from the Cochrane Collaboration). References are now also sourced from Google Scholar. Auto alerts have also been set up on these databases to ensure that the treatment studies catalogued on this website are regularly updated. Members of the speechBITE advisory committee also contributed key papers from their personal collections. In addition, speech pathologists from the University of Sydney and other academic institutions have also generously contributed treatment studies for inclusion in the database.
How does speechBITE decide which studies go on the website?
Treatment studies on speechBITE must fulfil the following criteria:
- The paper is published as a full-length paper (not an abstract) in a peer-reviewed scientific journal.
- The population treated will have, or be at risk of, a communication disorder and/or dysphagia.
- The treatment is currently part of speech pathology practice or could become part of speech pathology practice. The intervention need not be carried out by speech pathologists.
- The trial includes evaluation of at least one intervention and includes empirical data regarding treatment efficacy (i.e. editorials, non-systematic reviews or case descriptions are not included).
What is a clinical practice guideline?
Clinical practice guidelines are systematically developed statements that help clinicians and consumers make appropriate healthcare decisions. Such guidelines present statements or recommendations of ‘best practice’ based on a thorough evaluation of the evidence from published research studies on the outcomes of treatment (NHMRC, 2000).
Please note that speechBITE only started adding clinical practice guidelines in November 2013. The number of clinical practice guidelines will increase over time as more are identified which meet the speechBITE criteria.
How does speechBITE decide which clinical practice guidelines go on the website?
Clinical practice guidelines need to meet six criteria on order to be included on speechBITE. These criteria are based on those developed by the National Health and Medical Research Council clinical practice guideline portal. The six criteria are as follows:
- The clinical practice guideline was produced under the auspices of a health profession specialty association, health care organisation, relevant professional society, public or private organisation, government agency at the Federal, State, or local level. A clinical practice guideline developed and issued by an individual or group of individuals not officially sponsored or supported by one of the above types of organisations does not meet the inclusion criteria for speechBITE.
- A systematic literature search and review of existing scientific evidence published in peer-reviewed journals was performed during the guideline development OR the guidelines were based on a systematic review published within the five years preceding publication of the guideline.
- The clinical practice guideline contains systematically developed statements that include recommendations, strategies, or information that assists speech pathologists or clients to make decisions about appropriate health care for specific clinical circumstances.
- At least one specific recommendation in the guideline concerns at least one intervention that is currently part of speech pathology practice or that could become part of speech pathology practice.
- The clinical practice guideline was published or revised within the previous five years from the date it is listed on speechBITE.
- The clinical practical guideline is publicly available either by open/free access or by purchase.
The selection process for inclusion of clinical practice guidelines on speechBITE does not include an assessment of the methodological rigour, potential biases or accuracy of recommendations contained within the guidelines.
In what order does speechBITE list studies?
The search results you obtain on speechBITE are listed according to a level of evidence hierarchy. Whilst there are several such hierarchies available, speechBITE uses the current levels of evidence table from the National Health and Medical Research Council of Australia (NHMRC) in deciding which order to list study designs on speechBITE.
Level of evidence tables are a general guide to the quality of evidence, with the designs higher up in the hierrachy considered to have less risk of bias. However, the level of evidence does not necessarily equate to strength of evidence. This is why speechBITE provides methodological ratings on the PEDro-P scale for group comparison studies.
The following table lists the current NHMRC levels and provides an explanation of how the first four intervention study designs are indexed and listed on speechBITE.
|Type||Study Design||on speechBITE|
|Level 1||Systematic review of randomised controlled trials||All systematic reviews are listed first in search results. Systematic reviews are not rated.|
|Level 11||Randomised controlled trials||Randomised controlled trials are listed second in the search results. Randomised controlled trials are rated on the PEDro-P scale for methodological quality.|
|Level 111-1||Pseudo-randomised controlled trials (i.e. using alternate allocation or some other non-random method)||Indexed as non-randomised controlled trials and listed third on speechBITE. Non-randomised controlled trials are rated on the PEDro-P scale for methodological quality.|
|Level 111-2||A comparative study with concurrent controls: non-randomised experimental trial, cohort study, case-control study or interrupted time series||Indexed as non-randomised controlled trials and listed third on speechBITE. Non-randomised controlled trials are rated on the PEDro-P scale for methodological quality.|
|Level 111-3||A comparative study without concurrent controls: historical control study, two or more single arm study, interrupted time series without a parallel control group||Indexed as non-randomised controlled trials and listed third on speechBITE. Non-randomised controlled trials are rated on the PEDro-P scale for methodological quality.|
|Level 4||Case series with either post-test or pre-test/post-test outcomes||Case series with both pre-test/post-test outcomes are listed fourth on speechBITE. Case series are not rated for quality on speechBITE.|
Single-case study designs are not included in most level of evidence hierachies. Single-case research is a common design used in speech pathology research to study the individual effects of an intervention, especially in populations where randomised controlled trials are not feasible. A recent analysis of the speechBITE website found that single-case research was the most frequently used design for intervention research on speech, language and literacy disorders (Munro et al, 2013) (PDF 233 Kb). Likewise, a similar analysis of the PsycBITE database also found that single-case research was the dominant design used in 70% of aphasia therapy studies on this database (Togher et al, 2009).
In recognition of the relevance and importance of single-case research to clinical practice, speechBITE has chosen to include these studies in the database. It should be noted that only single-case studies with both pre and post treatment data are included in the database (i.e. case descriptions or case reports without pre/post data are excluded). One of speechBITE’s future goals is to assess the methodological quality of single-case research on the recently revised RoBiN-T scale.
Clinical practice guidelines are also listed on speechBITE as they are considered an important EBP resource for the translation of the best evidence into clinical practice.
How are studies rated?
Randomised controlled trials (RCTs) and non randomised controlled trials (NRCTs) are rated using the PEDro-P scale. Each treatment study is reviewed by two independent raters. If there is a discrepancy in the rating of any criteria then the paper is reviewed by a third rater to obtain consensus. Raters have been trained in the use of the PEDro-P scale and include members of the speechBITE advisory committee or other volunteer speech pathologists who have completed the rating training with the speechBITE team from The University of Sydney.
Which study designs are rated?
Currently speechBITE rates randomised and non-randomised controlled trials. Systematic reviews, case series and clinical practice guidelines are not rated according to quality on speechBITE. speechBITE plans to rate single-case experimental designs in the near future using the revised RoBiN-T scale.
Which ratings scales are used?
The PEDro-P scale is used to rate randomised and non-randomised controlled trials. speechBITE plans to rate single-case designs in the near future using the revised RoBiN-T scale.
What do I do if I don’t agree with a rating?
If you disagree with a rating of a treatment study on speechBITE, you can contact us. Please specify the authors, title and source of the study along with your reason for why you think the trial has been incorrectly rated. It is helpful if you can specify the location in the article (page and paragraph) where the supporting information can be found. All disputed ratings will be reviewed by the advisory committee.
Are the ratings on speechBITE reliable?
The speechBITE advisory committee have recently published an article (Murray et al, 2013) evaluating the reliability of the PEDro-P ratings on speechBITE. The speechBITE PEDro-P ratings ranged from fair to excellent for both the total score and for each of the 11 scale items. Furthermore, reliability was equal to that of similar allied health databases.
Can I become a rater for speechBITE?
speechBITE is fortunate to have a number of dedicated volunteers who rate papers on a regular basis. At present we are not able to train any more volunteer raters. If you would like to learn how to use the PEDro-P scale to rate randomised and non-randomised controlled trials yourself, speechBITE has a free online training program.
How do I…
Access full text for articles on speechBITE?
speechBITE does not provide the full text of journal articles. You need to note the papers that you wish to read (by choosing to print/email/save/export) and then either visit your local academic library and find the appropriate journal or access the on-line journal via your academic library website. Some professional associations and organisations provide access to on-line journals for members and staff. For example, members of Speech Pathology Australia have free online access to the International Journal of Speech-Language Pathology. Members of the American Speech-Language Hearing Association have on-line access to the Journal of Speech, Language and Hearing Research, American Journal of Speech-Language Pathology, Language Speech and Hearing Services in Schools and the American Journal of Audiology. Copyright permission is also required to publish the abstracts on speechBITE. If an abstract is not available on speechBITE, then either copyright permission is pending or it has been declined.
Report an indexing or rating error?
If you disagree with a rating of a particular study on speechBITE, contact us via email. Please specify the authors, title and source of the study along with your reason for why you think the study has been incorrectly rated. It is helpful if you can specify the location in the article (page and paragraph) where the supporting information can be found. All queries regarding ratings will be reviewed by the speechBITE committee and adjustments will be made based on a consensus decision made by the committee. On speechBITE each paper is indexed according to the following domains:
- Speech Pathology Practice Area
- Type of intervention
- Within this population
- Age group
- Type of service delivered
- Research design
If you believe that a paper has been indexed incorrectly, please let us know. Contact us via email and cut and paste the record from the “Search results” into the message. Please identify the type of indexing error, e.g. incorrect categorisation as “Group” under “Type of service delivered” instead of “Individual”, and please provide some reasoning or justification. The speechBITE committee will review the indexing of the paper and make adjustments as warranted.
Suggest publications for inclusion on speechBITE?
If you know of a study or paper that you think should be included on speechBITE, please first check that it meets the selection criteria for inclusion. If it does, please contact us via email with the reference details (authors, date, title and journal).
Get help with searching speechBITE?
speechBITE needs sponsorship to continue to be able to provide FREE access anywhere in the world to speech pathologists and communication specialists. We also need donations to help us continue to find and rate past and future research trials. Sponsorship of speechBITE will promote the use of evidence-based practice by health professionals and in doing so assist the wellbeing of all those with communication and swallowing difficulties. If you would like to sponsor the activities of speechBITE and make a difference please donate now.