• Moderation

Evaluating Ideas

What Are Kindling Idea Evaluations?

Idea Evaluations are a quick way to gather feedback about multiple Ideas from your network of Kindling participants. Evaluations enable you to look at Ideas comparatively and to make more informed decisions about which Ideas to move forward with.

How Do I Create an Evaluation?

As a Moderator, you can evaluate a set of Ideas within a Category/Campaign.

From the Category/Campaign screen, choose the “Moderate” option. Then choose “Create Evaluation”.

Evaluate Ideas

You can view and select all Ideas in the Category or Campaign, or filter the Ideas by vote count, state, tags, or keywords.

Filter Ideas

Select Ideas

Click “Next Step” to choose the evaluation type. Ranking allows recipients to rank Ideas according to criteria you specify. Scaling allows recipients to score ideas on a four point scale across multiple evaluation criteria. Ideas are then scored across all criteria and recipients and listed according to their score.

Once you have selected the Evaluation Type, click “Next Step” to enter your criteria for evaluating the selected Ideas.

For a ranking evaluation, enter the criteria that will inform recipients how they should rank the Ideas you have selected.

Ranking Criteria

For a Scaling evaluation, you will be prompted to enter the names of the evaluators who will analyze each Idea. Then enter the endpoints for the four point scale, where the lowest endpoint is the least desirable outcome and the highest endpoint is the most desirable outcome. For example, if your criterion is “How much time will it take to get this idea to market?” then your lowest endpoint will be far into the future (less desirable) and your highest endpoint will be in the near term as a quick implementation (best possible scenario).

Scaling Criteria

Once you have finished entering your criteria, click “Next Step”.

Enter basic information about the Evaluation including a title, a due date, and an optional message to the recipient(s), then click “Next Step”.

Add Basics

Select the recipients that will be evaluating the chosen Ideas. You can manually select the recipients or use the filter to search for certain recipients. Once you have selected the recipients, they will appear at the top under “Selected Users”.

Add Recipients

Click “Next Step” to review the selections you’ve made. To make changes, click the “Edit” link for that section. Once you are ready to send the Evaluation to the recipients, click “Send Evaluation Request”.

How Do I Reuse an Evaluation?

Moderators and Administrators can reuse or clone evaluations from the Manage Evaluations screen by clicking on the gear icon and selecting “Clone”.

Cloning Evaluations

Evaluation type, evaluation questions, criteria and messaging will all be pre-selected on your new evaluation, allowing you to reuse your carefully crafted questions and criteria to analyze new ideas and reach out to more experts for feedback.

What are some examples of evaluation criteria that I can use for Ranking or Scaling?

Evaluation criteria provide context on how recipients should analyze a group of Ideas. Some examples of evaluation criteria are: the cost or time to implement the Ideas, the impact the Ideas will have on the company, or the potential return on investment.

How Do I Know When an Evaluation Has Been Assigned to Me?

When you have been tasked with evaluating a select group of Ideas, you will receive an email notification that includes a link to the Evaluation and the due date. You will receive an in-app task notification as well. An alert icon will appear on your avatar and will also display beside the Tasks link indicating that you have an outstanding Task.

Task Notification

Clicking on ”Tasks” will take you to the page that displays all outstanding and completed Tasks assigned to you. On this page you will see the Evaluation task, the name of the person who requested the Evaluation, the due date, and a personal message if one was included.

Task View

How do I complete a Ranking Evaluation?

Click on the link in the Evaluation notification email or click on your avatar and select “Tasks”. Then, click on the title of the Evaluation.

To rank Ideas, drag them up or down using the dotted icon to the right of the Ideas. Drag the Idea you wish to rank as the highest to the top, with other Ideas following in a lower rank.

Rank Ideas

You can also click on the Idea title in the left pane to preview the number of votes, views, and comments associated with that Idea. Clicking on the Idea title in the right pane will allow you to view the Idea details in another window.

View Ideas to Rank

When you are finished ranking the Ideas, check the box for “I’m done ranking these Ideas”, then click the “Submit” button.

If you need to exit the evaluation before you are finished, Kindling will remember your choices and you can pick up where you left off when you return.

View Ideas to Rank

How do I complete a Scaling Evaluation?

Click on the link in the Evaluation notification email or click on your avatar and select “Tasks”. Then click on the title of the Evaluation.

Listed on the left will be a set of Ideas you have been asked to evaluate. On the right will be a series of questions with criteria which you will use to evaluate the Ideas.

Evaluate Ideas - Scaling

Read through each question and determine where you feel the Idea will fit among the presented criteria. To make your selection, click on the circle icon. Your selected choice will be denoted with a black dot inside of the circle.

Clicking on the Idea title in the right pane will allow you to view the Idea details in another window. If at any time you wish to return to the top of the list, click the “Back to Top” link located above the Idea.

When you are finished evaluating each Idea, click the “Submit” button.

Evaluate Ideas - Submit

If you need to exit the evaluation before you are finished, Kindling will remember your choices and you can pick up where you left off when you return.

How Do I View an Individual Evaluation Response?

You can view individual responses to Evaluations while they are still in progress and also when they are completed. Click on the “Manage Evaluations” tab then the “In Progress” or “Completed” tab of the related Category/Campaign.

Manage Evaluations

Next, click on the title of the Evaluation for which you‘d like to view individual responses. You can then click on each respondent to view their response.

Ranking Individual Evaluation Response

Scaling Individual Evaluation Response

How Do I View the Status of an In-progress Evaluation?

In-progress Evaluations appear in the “Manage Evaluations” tab of the related Category/Campaign. Click on the “In Progress” tab to see all evaluations in that Category.

In Progress Evaluation

The dashboard overview shows the Evaluation title, due date by which all responses should be returned, the date the Evaluation request was sent, and the name of the Moderator who requested the Evaluation. This view also shows the number of Ideas to be evaluated, as well as the number of recipients who have responded.

How Do I Know When an Evaluation Has Been Completed by All Participants?

When an Evaluation has been completed by all participants, the issuing Moderator will receive an email notification linking them to the full results of the Evaluation. Clicking on the link within the email will take the Moderator to the completed Evaluation.

An in-app notification will also appear. The bell icon in the upper-right corner of the application will have a red indicator when an Evaluation has been completed by all participants.

How Do I View a Completed Evaluation?

A Moderator can view a completed Evaluation by clicking on the link in the email they received when an Evaluation was completed, or by clicking on the link in the in-app notification.

Alternatively, Moderators may click on the “Manage Evaluations” tab of the related Category/Campaign and then click on the “Completed” tab to view all completed Evaluations.

Completed Evaluation

The dashboard view gives an overview of the Evaluation, including the Evaluation title, number of Ideas, number of respondents who completed the Evaluation, and the average of the ranked Ideas.

Clicking on the Evaluation title brings you to the complete Evaluation results. This page shows the title of the Evaluation, the respondents, and the final results of the Ideas that were ranked.

Evaluation Results

For a Scaling evaluation, the final results page will display the score for each criterion along with a total score per Idea. You can also drill down farther to view results for different scenarios, including specific respondents and questions.

You can view results for all questions and all respondents,

Scaling Results

focus on one particular question,

Scaling Results

or view results for a particular set of questions with just a few recipients.

Scaling Results

You may moderate any of the Ideas by checking the box next to the Idea and clicking “Moderate Ideas”. From here, you can begin making decisions on the Ideas if you’d like to Move Ideas to a New Category or Campaign, Change the State of Ideas, Comment on Ideas, or Add or Edit Tags.

Moderate Ideas

How Do I Share the Results of an Evaluation?

Evaluation results can be shared with Administrators or other Moderators assigned to this Category/Campaign to draw their attention to expert feedback and to bring them into the decision-making process. From the Manage Evaluations page, click on the title of the Evaluation you wish to share. Then, on the results page, click the gear icon to the right and select “Share”.

share evaluation

In the share dialog box, enter the recipient you would like to share the Evaluation results with. You may also enter multiple recipients.

Next, enter an optional message for the recipient and click the “Share” button.

share dialog evaluation

The recipients will receive an email and in-app notification indicating that you have shared a set of Evaluation results with them.

share evaluation in-app

How do I export the results of an Evaluation?

To export the results of an Evaluation, navigate to the Manage Evaluations page for the related Category/Campaign. Click the gear icon next to the title of the Evaluation you wish to export, and select “Export”.

Export Evaluation

The exported file will be downloaded as an Excel file which you can then view using a compatible program.

You can also export the results of an Evaluation while on the Evaluation Results page by clicking the gear icon and selecting “Export”.

Export Evaluation

How Do I Remove an Evaluation?

You can remove an Evaluation that is currently in progress or an Evaluation that has been completed. You may decide that you no longer want to continue evaluating Ideas for a particular Category or Campaign, and you may also want to remove completed evaluations once you have reviewed the results and made decisions on those Ideas.

To remove an Evaluation, click on the “Manage Evaluations” tab of the related Category/Campaign, then find the Evaluation you would like to remove. Next, click the gear icon and select “Remove Evaluation”.

Remove Evaluation

Are Users Reminded If a Due Date for an Evaluation Is Approaching?

Users will receive an email notification 2 days prior to the due date of the Evaluation if they have not yet completed their Evaluation. The notification will request that the User complete the Evaluation and will include the due date and the title of the Evaluation as well as a link to the requested Evaluation.

What If Not All Respondents Have Completed an Evaluation before the Due Date?

If a respondent has not completed an Evaluation and the due date for completion has passed, the Evaluation results will not include that response. The Evaluation results will indicate how many recipients responded out of the total number of recipients that were tasked with completing the Evaluation.