Feature Prioritization with Elote
Overview
Feature prioritization is a critical challenge in product management and software development. With limited resources and time, teams must decide which features to build first to maximize value. Traditional prioritization methods like scoring rubrics or the RICE framework (Reach, Impact, Confidence, Effort) have limitations:
They often rely on subjective numerical scores
Different stakeholders may interpret scoring criteria differently
It’s difficult to maintain consistency across many features
Comparing features directly is often more intuitive than scoring them individually
Elote offers an elegant solution through pairwise comparison-based ranking. By having stakeholders compare features directly against each other, you can build a more reliable and consensus-driven prioritization system.
This guide demonstrates how to use Elote for effective feature prioritization.
Basic Implementation
Let’s start with a simple implementation using the Elo rating system:
from elote import EloCompetitor
import json
class FeaturePrioritizer:
def __init__(self):
self.features = {}
def add_feature(self, feature_id, name, description, initial_rating=1500):
"""Add a feature to the prioritization system"""
if feature_id not in self.features:
self.features[feature_id] = {
'id': feature_id,
'name': name,
'description': description,
'competitor': EloCompetitor(initial_rating=initial_rating),
'comparisons': 0
}
return self.features[feature_id]
def record_comparison(self, winner_id, loser_id):
"""Record the result of a feature comparison"""
if winner_id not in self.features or loser_id not in self.features:
raise ValueError("Both features must be added to the system first")
# Update ratings
self.features[winner_id]['competitor'].beat(self.features[loser_id]['competitor'])
# Update comparison counts
self.features[winner_id]['comparisons'] += 1
self.features[loser_id]['comparisons'] += 1
def get_prioritized_list(self, min_comparisons=3):
"""Get a prioritized list of features"""
# Only include features with sufficient comparisons
valid_features = [f for f in self.features.values() if f['comparisons'] >= min_comparisons]
# Sort by rating (descending)
valid_features.sort(key=lambda f: f['competitor'].rating, reverse=True)
return [
{
'id': f['id'],
'name': f['name'],
'description': f['description'],
'rating': f['competitor'].rating,
'comparisons': f['comparisons']
}
for f in valid_features
]
def get_next_comparison(self):
"""Get the next most informative comparison to make"""
import random
# Simple strategy: find features with similar ratings
features_list = list(self.features.values())
if len(features_list) < 2:
return None
# Sort by number of comparisons (ascending)
features_list.sort(key=lambda f: f['comparisons'])
# Take the two least-compared features
if features_list[0]['comparisons'] < 5:
return (features_list[0]['id'], features_list[1]['id'])
# Otherwise, find features with similar ratings
features_list.sort(key=lambda f: f['competitor'].rating)
# Find adjacent features with closest ratings
min_diff = float('inf')
best_pair = None
for i in range(len(features_list) - 1):
diff = abs(features_list[i]['competitor'].rating - features_list[i+1]['competitor'].rating)
if diff < min_diff:
min_diff = diff
best_pair = (features_list[i]['id'], features_list[i+1]['id'])
return best_pair
# Usage example
prioritizer = FeaturePrioritizer()
# Add features
prioritizer.add_feature("f1", "User Authentication", "Allow users to sign up and log in")
prioritizer.add_feature("f2", "Dashboard", "Create a dashboard with key metrics")
prioritizer.add_feature("f3", "Export to CSV", "Add ability to export data to CSV")
prioritizer.add_feature("f4", "Dark Mode", "Add a dark mode theme option")
prioritizer.add_feature("f5", "Mobile App", "Create a mobile app version")
# Record some comparisons
prioritizer.record_comparison("f1", "f3") # Auth is more important than CSV export
prioritizer.record_comparison("f1", "f4") # Auth is more important than Dark Mode
prioritizer.record_comparison("f2", "f3") # Dashboard is more important than CSV export
prioritizer.record_comparison("f2", "f4") # Dashboard is more important than Dark Mode
prioritizer.record_comparison("f5", "f4") # Mobile App is more important than Dark Mode
prioritizer.record_comparison("f1", "f2") # Auth is more important than Dashboard
prioritizer.record_comparison("f5", "f3") # Mobile App is more important than CSV export
prioritizer.record_comparison("f1", "f5") # Auth is more important than Mobile App
# Get prioritized list
prioritized = prioritizer.get_prioritized_list()
for i, feature in enumerate(prioritized, 1):
print(f"{i}. {feature['name']} (Rating: {feature['rating']:.1f})")
Collecting Comparison Data
There are several ways to collect feature comparison data:
Stakeholder Voting: Have stakeholders vote on pairs of features
Team Meetings: Conduct structured prioritization sessions
User Research: Ask users to compare feature importance
Automated Surveys: Send regular comparison questions to stakeholders
Here’s an example of a simple web interface for collecting stakeholder votes:
<div class="feature-comparison">
<h2>Which feature is more important?</h2>
<div class="feature-options">
<div class="feature" onclick="recordPreference('f1')">
<h3>User Authentication</h3>
<p>Allow users to sign up and log in</p>
</div>
<div class="feature" onclick="recordPreference('f2')">
<h3>Dashboard</h3>
<p>Create a dashboard with key metrics</p>
</div>
</div>
</div>
<script>
function recordPreference(preferredFeatureId) {
const otherFeatureId = preferredFeatureId === 'f1' ? 'f2' : 'f1';
// Send data to your backend
fetch('/api/record-preference', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
winner: preferredFeatureId,
loser: otherFeatureId,
voter: currentUserId,
timestamp: new Date().toISOString()
})
});
// Load the next comparison
loadNextComparison();
}
</script>
Advanced Implementation: Multi-Criteria Prioritization
In real-world scenarios, features are often evaluated on multiple criteria such as:
Business value
User value
Implementation effort
Technical risk
Strategic alignment
We can use Elote’s Ensemble rating system to combine these criteria:
from elote import EnsembleCompetitor, EloCompetitor
class MultiCriteriaFeaturePrioritizer:
def __init__(self, criteria=None, weights=None):
self.features = {}
# Default criteria and weights if not provided
self.criteria = criteria or ["business_value", "user_value", "effort", "risk"]
self.weights = weights or [0.4, 0.3, 0.2, 0.1]
# Validate weights sum to 1
if abs(sum(self.weights) - 1.0) > 0.001:
raise ValueError("Weights must sum to 1.0")
if len(self.criteria) != len(self.weights):
raise ValueError("Must provide a weight for each criterion")
def add_feature(self, feature_id, name, description):
"""Add a feature to the prioritization system"""
if feature_id not in self.features:
# Create a competitor for each criterion
rating_systems = []
for criterion, weight in zip(self.criteria, self.weights):
rating_systems.append((EloCompetitor(initial_rating=1500), weight))
self.features[feature_id] = {
'id': feature_id,
'name': name,
'description': description,
'competitor': EnsembleCompetitor(rating_systems=rating_systems),
'comparisons': {criterion: 0 for criterion in self.criteria},
'total_comparisons': 0
}
return self.features[feature_id]
def record_comparison(self, winner_id, loser_id, criterion=None):
"""
Record a comparison result
If criterion is None, update the ensemble rating
If criterion is specified, update only that criterion's rating
"""
if winner_id not in self.features or loser_id not in self.features:
raise ValueError("Both features must be added to the system first")
winner = self.features[winner_id]
loser = self.features[loser_id]
if criterion is None:
# Update the ensemble rating
winner['competitor'].beat(loser['competitor'])
winner['total_comparisons'] += 1
loser['total_comparisons'] += 1
elif criterion in self.criteria:
# Update a specific criterion
criterion_index = self.criteria.index(criterion)
# Get the specific rating system
winner_rating_system = winner['competitor'].rating_systems[criterion_index][0]
loser_rating_system = loser['competitor'].rating_systems[criterion_index][0]
# Update the rating
winner_rating_system.beat(loser_rating_system)
# Update comparison count for this criterion
winner['comparisons'][criterion] += 1
loser['comparisons'][criterion] += 1
else:
raise ValueError(f"Unknown criterion: {criterion}")
def get_prioritized_list(self, min_comparisons=3):
"""Get a prioritized list of features"""
# Only include features with sufficient comparisons
valid_features = [
f for f in self.features.values()
if f['total_comparisons'] >= min_comparisons
]
# Sort by ensemble rating (descending)
valid_features.sort(key=lambda f: f['competitor'].rating, reverse=True)
result = []
for f in valid_features:
# Get individual criterion ratings
criterion_ratings = {}
for i, criterion in enumerate(self.criteria):
rating_system = f['competitor'].rating_systems[i][0]
criterion_ratings[criterion] = rating_system.rating
result.append({
'id': f['id'],
'name': f['name'],
'description': f['description'],
'overall_rating': f['competitor'].rating,
'criterion_ratings': criterion_ratings,
'comparisons': f['comparisons'],
'total_comparisons': f['total_comparisons']
})
return result
Handling Stakeholder Disagreement
Different stakeholders often have different priorities. Here are strategies to handle this:
Separate Rankings: Maintain separate rankings for different stakeholder groups
Weighted Voting: Give different weights to different stakeholders
Consensus Building: Use the comparison process to drive discussion and consensus
Here’s an example of weighted voting:
def record_weighted_comparison(self, winner_id, loser_id, voter_id, voter_weights=None):
"""Record a comparison with different weights for different voters"""
if voter_weights is None:
# Default weights by role
voter_weights = {
"executive": 2.0,
"product_manager": 1.5,
"engineer": 1.0,
"designer": 1.0,
"sales": 0.8
}
# Get voter's role
voter_role = self.get_voter_role(voter_id)
weight = voter_weights.get(voter_role, 1.0)
# Apply the comparison multiple times based on weight
full_weight = int(weight)
for _ in range(full_weight):
self.record_comparison(winner_id, loser_id)
# Handle fractional weight
fractional_weight = weight - full_weight
if random.random() < fractional_weight:
self.record_comparison(winner_id, loser_id)
Visualizing Prioritization Results
Visualization helps stakeholders understand and buy into the prioritization results:
import matplotlib.pyplot as plt
import numpy as np
def visualize_prioritization(prioritized_features, top_n=10):
"""Create a visualization of the prioritization results"""
# Take top N features
features = prioritized_features[:top_n]
# Extract data
names = [f['name'] for f in features]
ratings = [f['rating'] for f in features]
# Create horizontal bar chart
plt.figure(figsize=(10, 8))
y_pos = np.arange(len(names))
plt.barh(y_pos, ratings, align='center')
plt.yticks(y_pos, names)
plt.xlabel('Rating')
plt.title('Feature Priority Ranking')
# Add rating values at the end of each bar
for i, v in enumerate(ratings):
plt.text(v + 10, i, f"{v:.1f}", va='center')
plt.tight_layout()
plt.savefig('feature_prioritization.png')
plt.close()
return 'feature_prioritization.png'
Real-World Applications
Organizations using pairwise comparison for feature prioritization include:
Microsoft: Uses similar techniques for Windows feature prioritization
Atlassian: Incorporates pairwise voting in their product planning
Spotify: Uses comparison-based methods for roadmap planning
IBM: Employs similar techniques for enterprise software features
Best Practices
Start Small: Begin with a manageable set of features (15-20)
Involve Key Stakeholders: Ensure all perspectives are represented
Provide Context: Give voters sufficient information about each feature
Regular Updates: Re-prioritize regularly as new information emerges
Combine Methods: Use Elote alongside other prioritization frameworks
Document Decisions: Record the reasoning behind key prioritization decisions
Consider Dependencies: Factor in technical dependencies between features
Conclusion
Elote provides a powerful framework for implementing feature prioritization systems based on pairwise comparisons. This approach offers several advantages over traditional scoring methods:
More intuitive for stakeholders (comparing is easier than scoring)
Reduces individual bias through multiple comparisons
Creates a clear, defensible ranking
Adapts as new features are added
Builds consensus through the comparison process
By leveraging Elote’s implementation of sophisticated rating systems, you can build robust feature prioritization solutions that help your team focus on delivering the highest-value features first.