Grading Rubrics

posted in: Newsletter, Teaching | 0

sample levels for a rubric, excellent, very good, good, average, poorA rubric is a tool that lists evaluation criteria for an assignment. Rubrics can help students organize their efforts to meet the requirements of an assignment and faculty can use rubrics to explain their evaluations to students.

Rubrics are made up of rows and columns. The rows correspond to the various criteria of an assignment and the columns correspond to the level of achievement expressed for each criterion. A description and point value for each cell in the rubric defines the evaluation and score of an assignment.

 

Simple Grading Rubric for a History Research Paper

Excellent Good Poor
3 2 1
Number of sources 10 – 12 5 – 9 1 – 4
Historical accuracy No apparent inaccuracies Few inaccuracies Lots of historical inaccuracies
Organization Can easily tell from which sources information was drawn Can tell with difficulty from where information came Cannot tell from which source information came
Bibliography All relevant bibliographic information is included Bibliography contains most relevant information Bibliography contains very little information

 

Basic Steps in Creating a Rubric

  1. Select a Performance/Assignment to be Assessed.
    Performances and assignments which may be difficult to grade and where you want to reduce subjectivity are great candidates for incorporating rubrics. Is the performance/assignment an authentic task related to learning goals and/or objectives? Are students replicating meaningful tasks found in the real world? Are you encouraging students to problem solve and apply knowledge? Answer these questions as you begin to develop the criteria for your rubric.Performance example: Writing a Research Paper on a Topic Related to Local History
  2. Identify the Criteria.
    Create a list of all traits, features or dimensions that you want to measure, and include a definition and example to clarify the meaning of each trait being assessed. Each assignment or performance will have its own unique traits to be scored. Then reduce the list by chunking similar criteria and eliminating others until you produce a range of appropriate criteria. Keep the list manageable and reasonable.Criteria examples for a term paper:
    Introduction
    Thesis statement
    Arguments/analyses
    Grammar and punctuation
    Spelling
    Internal citations
    Conclusion
    Reference page
  3. Set the Point Value.
    Point values make up the system of numbers or values used to rate each criterion and often are combined with levels of performance. Make sure the values make sense in terms of the total points possible: Is there a difference between getting 10 points versus 100 points versus 1,000 points? The best and worst performances are placed at the ends of the continuum and the other scores are placed appropriately in between. It is best to start with fewer levels and to distinguish between work that doesn’t meet the criteria.Point value examples: 1, 2, 3, 4, 5 or 2, 4, 6, 8
  4. Write the Descriptions.
    Descriptions spell out each level (gradation) of performance for each criterion and describe what performance at a particular level looks like. Descriptions describe how well student’s work is distinguished from the work of their peers and will help you to distinguish the differences between students’ work.Description examples in italics:

    Criterion

    Excellent
    5

    Good
    4

    Fair
    2

    Poor
    1

    Spelling No spelling errors. One or two spelling errors, but not of the type to make meaning obscure, and not of basic or common words A few minor spelling errors (more than two) but not enough to harm your ethos seriously or impede a reader’s comprehension. Many misspelled important or common words, or a number of minor errors that interfere with easy reading or comprehension.

     

  5. Determine Levels of Performance.
    Select words or phrases (often as adjectives) that will explain what performance looks like at each level, making sure they are discrete enough to show real differences. Levels of performance should match the related criterion.Levels of performance examples:
    Excellent, Good, Fair, Poor
    Master, Apprentice, Beginner
    Exemplary, Accomplished, Developing, Beginning, Undeveloped
    Complete, Incomplete
    Yes, No
  6. Evaluate the Rubric.
    Evaluate the rubric each time it’s used to ensure it matches instructional goals and objectives. Be sure students understand each criterion and how they can use the rubric to their advantage. Review the rubric with a colleague, pilot test new rubrics if possible, and solicit students’ feedback for further refinements.

Types of Rubrics

Determining which type of rubric to use depends on what and how you plan to evaluate. There are several types of rubrics including holistic, analytical, general, and task-specific.

Holistic — all criteria are assessed as a single score which can be used for evaluating overall performance on a task. Because only one score is given, holistic rubrics tend to be easier to score. However, holistic rubrics do not provide detailed information on student performance for each criterion; the levels of performance are treated as a whole.

Analytical — each criterion is assessed separately, using different descriptive ratings, and thus, receive a separate score. Analytical rubrics take more time to score but provide more detailed feedback.

General — these rubrics can be used for similar performances such as a rubric for all final presentations, a rubric for all dance performances, or a rubric for all research proposals. Criteria are assessed separately, as in an analytical rubric.

Task-specific — are designed to assess a specific task in which each criteria is assessed separately. It may not be possible, however, to account for each and every criteria involved in a particular task which could overlook a student’s unique solution (Arter & McTighe, 2001).

Using Rubrics in Blackboard

You can build interactive rubrics in Blackboard to simplify the process of grading student work and returning rubric results to students. Interactive rubrics can be used with nearly every assessment method in Blackboard, including Assignments (including with SafeAssign enabled), short answer Test questions, and any Blogs, Journals, Wikis, or Discussion Board threads and forums that have grading enabled. Visit the Teaching with Blackboard website for instructions on using Blackboard’s Interactive Rubrics as well as tutorials, archives of online rubric workshops, and Quick Guides on using Rubrics in your teaching: http://www.niu.edu/blackboard/assess/rubrics.shtml

Summary

Grading rubrics are effective and efficient tools which allow for objective and consistent assessment of a range of performances, assignments, and activities. Rubrics clarify your expectations and will show students how to meet them, making them accountable for their performance in an easy-to-follow format. The feedback that students receive through a grading rubric can help them improve their performance on revised or subsequent work. Rubrics can also help to rationalize grades when students ask about your method of assessment. Rubrics also allow for consistency in grading for instructors who team-teach the same course, for TAs assigned to the task of grading, and can serve as good documentation for accreditation purposes. Finally, rubrics can reduce grading time, increase objectivity and reduce subjectivity, convey timely feedback to students, and improve students’ ability to include required elements of an assignment.

Free Rubric Builders and Generators.

Consider using any of free existing rubrics available online. Many rubrics can be used “as is” or can be modified to meet your specific needs. Creating a rubric from scratch will take time but may be necessary for a particular assignment. The following are school-based but are highly applicable to higher education.

Build a Rubric by Annenberg Learner
http://www.learner.org/workshops/hswriting/interactives/rubric/index.html

General Rubric Generator by teAchnology
http://www.teach-nology.com/web_tools/rubrics/general/

Create a Rubric by RubiStar
http://rubistar.4teachers.org/index.php

References and Resources

Arter, J., & McTighe, J. (2001). Scoring rubrics in the classroom: Using performance criteria for assessing and improving student performance. Thousand Oaks, CA: Corwin Press, Inc.

Blackboard Portfolio Tool: Faculty Perspectives

Blackboard Portfolio ImageDuring the summer of 2015, the Faculty Development and Instructional Design Center team worked collaboratively with many faculty and staff to prepare for the Fall 2015 implementation of the new Blackboard Portfolio tool. Blackboard portfolios were created in a variety of courses including UNIV 101 and First Year Composition courses, as well as other undergraduate and graduate level courses and programs.

The easy-to-use portfolio tool allows students and faculty to create several portfolios for different purposes. For example, portfolios may be created for a specific course, program of study, or job search.  Faculty can request templates to serve as a guide for students and grade the portfolio assignment in the Blackboard Grade Center. Students can upload completed assignments, projects from co-curricular activities, and career information using the portfolio tool. Students can then share their portfolio by submitting it to a course or via email to NIU users or non-NIU contacts with only an email address.

NIU faculty who have used the Blackboard Portfolio tool found it beneficial for their students. Some of the comments from faculty include:

It was easy to set-up.

 

The portfolio gave me (as instructor) far deeper insights…what really had an impact on their learning and how beneficial different teaching strategies or activities were from the students’ perspective.

 

For the students, I believe this portfolio really helped them explore and recognize everything that they accomplished in a very short time frame.

 

The portfolio also helped the students reflect on what type of learner they are…and how to adjust their approach to their own education and learning in the future.

 

It is so important to have all of the resources…in one place and this functionality in Blackboard made it especially convenient.

 

Students were able to include a variety of media (ppt, animation, audio, documents, links, etc.).

 

I was able to create a checklist/rubric to go along with the portfolio assignment, which made grading painless.

 

I was very pleased with the pilot of the e-portfolio. I would like to experiment with setting up an e-portfolio for another class I am teaching this semester.

We have extensive information and tutorials on using the Blackboard Portfolio tools at go.niu.edu/portfolios, including tutorials specifically for faculty and students to help you get started using Blackboard Portfolios. Contact us at facdev@niu.edu for more information on using the Blackboard Portfolio for your course or departmental ePortfolios!

Opportunity to Provide Feedback on Peer Assessment in Blackboard

posted in: News | 0

Two students working togetherBlackboard is in the early stages or investigating an integrated approach to peer assessment and would like to understand more about how faculty incorporate peer evaluations into their learning activities. They are looking for any feedback on how you currently use peer evaluation, whether that is conducted through Blackboard, other technology, or on paper. Your feedback will help Blackboard determine how the use of tools can help facilitate peer review and how those reviews may be incorporated into grading activities.

If you use peer evaluation as part of your teaching, please complete the short survey available at https://bbuxresearch.wufoo.com/forms/peer-assessment/. They survey should only take 10-15 minutes.

Tips to Reduce the Impact of Cheating in Online Assessment

Student sitting at computer

Preserving the integrity of student learning assessment is as much of a priority for online courses as it is for traditional face-to-face instruction. Although there is concern that academic dishonesty or ‘cheating’ might be more likely to happen in an online setting, studies comparing face-to-face and online settings have yielded mixed results (Grijalva, Nowell, Kerkvliet, 2006; Lanier, 2006; Stuber-McEwen, Wiseley, Hoggatt, 2009). Yet, a perception persists that challenges to preventing cheating are somewhat different in an online setting because faculty and students are physically separated from each other. This remoteness certainly makes it difficult to monitor various types of learning assessment activities.

Watson and Sottile (2010) have reported that cheating online can manifest in a number of ways including students who: (1) submit others’ work as their own, (2) receive/send answers from/to other students (by texting, using instant messaging, talking on a phone) while taking a test or quiz, (3) receive answers from a student who has already taken a test or quiz, (4) copy other students’ work without permission, (5) knowingly plagiarize from an article or book, or (6) use a term paper writing service.

Although it may be difficult to prevent cheating entirely, faculty can implement steps to reduce its impact in the student learning assessment process for online courses. The following are some practical tips to prevent or reduce cheating for two common learning assessment activities, namely testing and homework assignments.

Tips for Testing

    • Purposefully Select Assessment Methods – Use online testing, particularly objective test (i.e., multiple choice, multiple answer, true/false) for lower stakes assessment of student learning. In assessing student mastery of course goals and objectives, objective tests should be only one option considered among a spectrum of methods considered. Each type of assessment method may be designed to measure different indicators of student learning based on course goals and objectives. While an objective test can measure a student’s ability to recall or organize information, other methods can be used to assess higher order/critical thinking skills including understanding, applying, analyzing, evaluating, and creating (Krathwohl, 2001).

 

    • Mix Objective and Subjective Questions – While online testing can include objective measures (multiple choice, multiple answer, true/false, fill in the blank, etc.), faculty can also include short answer or essay questions. This type of question is more subjective in nature and may demand a deeper understanding of the subject being tested. While mixing objective and subjective type questions may not discourage or stop sharing of information, it may limit the effect on the student’s final grade (Watson and Sottile, 2010).

 

    • Use Question Pools – Rather than using a fixed number of items that remain unchanged for each administration of the test, consider creating a question pool. Questions can be grouped by any number of criteria, including topic, subject matter, question type or difficulty of question. A pool will generate an assessment with randomized questions selected by the faculty member. Pools can be created from new questions or questions in existing tests or pools. Pools are most effective when there are large numbers of questions in one group. For example, one might have a pool of true/false questions, another of multiple choice and a third for fill in the blank. The faculty member could then create an assessment drawing a specific number of questions from each of the question-type pools. Faculty can also add new questions to pools each time the course is taught to expand the variability of questions. Conversely, older questions can be removed. Check the Teaching with Blackboard Question Pools web page for more details.

 

    • Randomize Questions – When creating a test in Blackboard, one test option allows faculty to randomize the selection of test questions as well as the order in which they appear. The result is that students are not likely to get the same questions in the same sequence when taking a test. This strategy can address the issue of students who take a test at the same time in order to share answers. This is also relevant if faculty allow students to repeat the test. Each time this occurs, a test will be made up of questions that are randomly selected and ordered.

 

    • Limit Feedback – Limit what types of feedback is displayed to students upon completion of a test. Available test options include test ‘Score’, ‘Submitted Answers’, ‘Correct Answers’, and ‘Feedback’. Providing test scores is important feedback that indicates how well students have performed and should be made available. However, through a process of elimination, students may be able to determine the correct answer for each test question if their submitted answers are identified as incorrect, or if the correct answer is provided. Students could lose the incentive to both prepare for testing or to seek out correct answers by reviewing lecture notes, assigned readings, or through group discussion after completing tests. Thus, faculty might reconsider whether to include ‘Submitted Answers’ as an option to be displayed to students. This is especially relevant if faculty have allowed students to repeat tests. Each time a test was taken, students could attempt a different answer for a test question that was previously graded as incorrect. Correct answers to all test questions could eventually be accumulated and passed on to other students, or to students of future classes.

 

    • Set Timer – Recognizing the fact that students taking an exam that is not proctored are free to use open book/notes, faculty may decide to use the ‘Set Timer’ feature in Blackboard. Students who adequately prepared for a test may be less likely to rely on open book/notes compared with students unprepared for testing. By setting a test with an expected completion time, unprepared students could have the most to lose as they spend time going over material, and risk not having sufficient time to respond to all the test questions.

 

    • Display Questions One At A Time – If a test has more than 5 questions, do not choose the ‘All at Once’ option for displaying all the questions on the same screen. It is quite easy for students to take a screen capture of the displayed questions and share them with other students. While students can still screen capture pages with single questions, or even type them into a document, it is more time consuming and unwieldy.

 

Tips for Homework Assignments

    • Create Application Assignments – Create assignments that require students to apply essential course concepts to a relevant problem. This may force students to seek relevant information beyond the assigned readings and lectures, and conduct independent research by identifying credible sources to support the development of their assignments. Students can be required to report their progress on a regular basis through email, or through the Journal feature in Blackboard. This documentation makes it easier for faculty to see the development of a student’s work from inception to completion, and possibly identify unexplained gaps that could occur if students used the work of others and claimed it as their own. Faculty can add input at any point in this process to provide guidance, and perhaps suggest new directions for students. Both documentation of progress through regular status reporting and occasional faculty input can add a greater level of scrutiny to students, making it more difficult to pass off the work of others as their own.

 

    • Create Group Assignments – Create group assignments that require students to interact with group members regularly. Groups can be made responsible for determining the functional roles for each member, establishing a mechanism for accountability (i.e., submitting weekly progress reports), and sharing drafts of individual progress on a group project. For a project to be truly collaborative, each group member should be familiar with everyone else’s work, and be able to describe how every group members’ contribution supports the whole group assignment. Students who are using the work of others may not be able to adequately describe the significance of their ‘own’ work, or how it integrates with the group’s overall project.

 

    • Create Assignments that Require Presentations – Faculty with a Blackboard course can use the web conferencing tool, Blackboard Collaborate, to conduct a synchronous online session for class presentations. Students may be asked to submit a progress report or use a Journal to reflect on what they have learned in the past week that supports work toward the presentation. To further scrutinize work on the presentation, students may be asked to include time for questions and answers. Students who have developed the presentation should be comfortable answering a range of topic-related questions. Faculty Developments offers workshops and one-on-one consultations on the use Blackboard Collaborate.

 

    • Check for Plagiarism using SafeAssignSafeAssign is a plagiarism prevention tool that detects matches between students’ submitted assignments and existing works by others. These works are found on a number of databases including ProQuest ABI/Inform, Institutional document archives, the Global Reference Database, as well as a comprehensive index of documents available for public access on the internet. SafeAssign can also be used to help students identify how to attribute sources properly rather than paraphrase without citing the original source. Thus, the SafeAssign feature is effective as both a deterrent and an educational tool. You can learn more about SafeAssign by visiting the Faculty Development and Instructional Design Center’s Teaching with Blackboard SafeAssign Web page. Faculty Development also offers workshop and one-on-one consultations to faculty and staff on using SafeAssign.

 

    • Use Discussion Assignments – Create a Discussion Board assignment that requires students to demonstrate critical thinking skills by responding to a relevant forum topic. Faculty may also design a rubric that is specific to the Discussion Board assignments, and develop questions that require students to respond to every rubric category. Having assignments that are very specific makes it more difficult for students to use portions of a previous term paper or other sources that may only indirectly touch on the Discussion Board topic.

 

    • Include Academic Integrity Policy Statement in the Course Syllabus – Faculty should consider including a policy statement regarding academic integrity in their course syllabus. In addition, faculty may want to reiterate academic policies to students taking an online course and clarifying guidelines for completing test and assignments so that students are not confused about what they can and cannot do. While this, in and of itself, may not be sufficient to change behavior, its acknowledged presence in the syllabus recognizes a commitment to honesty in the academic arena and establishes the clear expectation that academic integrity is an important principle to live by. Faculty may also choose to mention this policy using the ‘Announcements’ feature in Blackboard, or while conducting a live web conference session.

 

Learn More

Preserving academic integrity is an ongoing challenge for traditional face-to-face, blended, and entirely online courses. While a number of expensive technology solutions, such as retinal eye scanners and live video monitoring are being developed to prevent cheating in online courses, the practical suggestions offered above can prevent or reduce the impact of cheating on assessing student performance in online courses. For more information on this topic, readers are invited to view the archived online workshop, “Tips for Assessing Student Learning Using Blackboard.”

In addition, the Faculty Development and Instructional Design Center offers many Blackboard workshops, including those that touch on assessment. You are invited to visit the Faculty Development Web site that lists upcoming faculty and staff programs. Finally, readers are encouraged to review the following suggested readings and resources.

Suggested Readings and Resources

Grijalva, T., Nowell, C., & Kerkvliet, J. (2006). Academichonesty and online courses. College Student Journal, 40(1), 180-185.

Hi-Tech Cheating: Cell Phones and Cheating in Schools – A National Poll. Common Sense Media. Retrieved from http://msnbcmedia.msn.com/i/MSNBC/Sections/NEWS/PDFs/2010_PDFs/100202_CellPhoneSchoolCheating.pdf.

Krathwohl, D. R. (2001). A revision of Bloom’s taxonomy: An overview. In L.W. Anderson & D.R. Krathwohl (Eds.), A taxonomy for learning, teaching, and assessing: A revision of Bloom’s Taxonomy of Educational Objectives. New York: Longman.

Lanier, M. (2006). Academic integrity and distance learning. Journal of Criminal Justice Education, 17(2), 244-261.

Luce, A. (2012, Sept. 17). How do I know students aren’t cheating? Instructional Design & Development Blog. Retrieved from http://www.iddblog.org/?p=1194

McNett, M. (2002). Curbing academic dishonesty in online courses. Pointers and Clickers: ION’s Technology Tip of the Month. Retrieved from http://www.ion.uillinois.edu/resources/pointersclickers/2002_05/index.asp.

Olt, M. R., (2002). Ethics and distance education: Strategies for minimizing academic dishonesty in online assessment. Online Journal of Distance Learning Administration, 5(3), Fall 2002.

Redmann, E. (n.d.) How Technology Is Raising the Stakes In Classroom Cheating. Edudemic. Retrieved from http://edudemic.com/2013/01/technology-classroom-cheating.

Stuber-McEwen, D., Wiseley, P., & Hoggatt, S. (2009). Point, click, and cheat: Frequency and type of academic dishonesty in the virtual classroom. Online Journal of Distance Learning Administration, 12(3), 1-10.

Trenholm, S. (2006-2007). A review of cheating in fully asynchronous online courses: A math or fact-based course perspective. Journal of Educational Technology Systems, 35(3), 281-300.

Watson, G., & Sottile, J. (2010). Cheating in the digital age: Do students cheat more in online courses? Online Journal of Distance Learning Administration. Retrieved from http://www.westga.edu/~distance/ojdla/spring131/watson131.html?goback=%2Egde_52119_member_208797940.

Increase the Fun in Your Courses with Digital Badges

Intro to Badges
A common concern in university courses is student motivation and engagement. There are many approaches to overcoming these issues, including Problem-Based Learning, Experiential Learning, and Service Learning. Game-Based Learning is also gaining popularity. Game-Based Learning incorporates elements of game design like mastering content or skills, overcoming challenges, earning points, and competing with others, into the learning process. Games are exciting, interesting, and motivating in ways that many courses are not.

Badges Defined

Digital badges, which are similar to Boy or Girl Scout merit badges, are a new way to add game-like elements to your course. Essentially, badges are digital artifacts that recognize an individual for learning or mastering a new skill. This could be the result of a formal learning experience, such as taking a university course, or something more informal, such as taking a community education course or belonging to a club, group, or other organization.

Depending on their implementation, badges can serve one or more of five social psychological functions (Antin & Churchill, 2011): goal setting, instruction, reputation, status/affirmation, and group identification. By defining goals for students, badges can motivate students to achieve the goal. Badges also provide instruction about the types of activities and social norms expected, particularly for students new to a field or system. Badges visually convey a student’s reputation within the system and provide information about their skills and expertise. Because badges also serve as a reminder of achievement, they also serve as a personal affirmation of past success, like a trophy on display. Finally, when badges reward a set of shared activities, badge ownership indicates group membership and can create a sense of solidarity among members.

Uses

Merit Badges
Photo courtesy Girl Guides of Canada

Many social websites have made use of badges, including Foursquare, Codecademy, and Khan Academy, to motivate users and increase use of the site. Some MOOCs (Massive Open Online Courses) use badges for recognition and achievement. In many cases, like the social websites, the badges are contained within the system and can only be viewed or shared internally. However, many systems are adopting the Mozilla Open Badges framework that puts the student in control of the badges. The open framework allows the learner to decide which badges to display and how they want to share them. In this way, badges can supplement a traditional resume or portfolio.

Course Implementation

The first step to implementing badges in a course is to define outcomes, goals, or skills that are significant. Many badge systems reward incremental progress rather than only completion, so it is possible to break large goals into smaller milestones. Based on the five social psychological functions, consider including badges that are not tied to mastering specific knowledge or skills. For instance, students could earn a badge for getting started, like submitting a first assignment or demonstrating knowledge of the syllabus and course structure. Badges could also be effective for encouraging and rewarding contributing to the course community, like taking the lead in a group project or being active on a course discussion board or blog.

Once the outcomes have been identified, create the images for the badges. Generally, badges have to be a .png image file. The image files are usually square and range from 150×150 pixels to 260×260 pixels (the final size will depend on the system used to deploy the badges). Be creative with the badge design. Most badge graphics are round, but other shapes, like shields, stars, and award ribbons are common as well. The image can include shapes, icons, photos, and text. Any graphics editing program can be used to create badges, like Adobe Photoshop, Inkscape, or pixlr. It is even possible to build the image on a PowerPoint slide, then save as an image (by right-clicking).

Finally, decide how to deploy and distribute the badges. One of the easiest systems to use is badg.us. It is a free service that establishes a redemption code for each badge. Students redeem the code to claim the badge. Purdue University is developing Passport, a learning system that demonstrates academic achievement through customizable badges. At the time of this writing, Passport is in beta but looks promising. It includes a badge-builder to create the graphic as well as student tracking to determine who has earned the badge.

In January, 2012, the Faculty Development and Instructional Design Center offered its first badge for participating in the Teaching Effectiveness Institute on Teaching in the Digital Classroom. The response was very positive.

badge for intro to badges
Intro to Badges

Non-traditional credentialing, through MOOCs and other means, is gaining popularity. The digital badge movement is part of the overall trend towards more granular and less formal methods of demonstrating competency. It is easy to get started with awarding badges for courses or organizations, and it may even add some fun to the experience!

Note: You have earned your first badge by reading this article! Click here to redeem your badge: http://badg.us/en-US/badges/claim/rw7kwa. (You will need to create a free badg.us account to accept the badge.)

Suggested Readings

Educause. (June 11, 2012). 7 Things you should know about badges. Retrieved from http://www.educause.edu/library/resources/7-things-you-should-know-about-badges

Hickey, D. (October 30, 2012). Introducing digital badges within and around universities. Retrieved from http://remediatingassessment.blogspot.com/2012/10/introducing-digital-badges-within-and.html

Mozilla. (2012). What are open badges? Retrieved from http://openbadges.org/en-US/

Young, J. (October 14, 2012). Grades out, badges in. Retrieved from http://chronicle.com/article/What-If-We-Got-Rid-of-Grades-/135056/

References

Antin, J., & Churchill, E. (2011). Badges in social media: A social psychological perspective. Paper presented at the CHI 2011, Vancouver, BC, Canada. Retrieved from http://research.yahoo.com/pub/3469

1 2