Skip to main content
Back to Work

AI-Powered Grant Proposal Evaluation System

Designed and developed an AI-assisted decision-support prototype to evaluate grant proposals against donor-specific requirements, combining Retrieval-Augmented Generation, Google Gemini, deterministic weighted scoring, and downloadable PDF reports.

Applied AIProduct Design · Technical Product Development · Product Evaluation
Project
Final-year Computer Science project
Institution
Usmanu Danfodiyo University, Sokoto
Product type
AI-assisted decision-support application
Case Study Contents (14)
01

Project overview

Grant proposal evaluation requires reviewers to examine lengthy applications against funding guidelines, eligibility requirements and multiple assessment criteria. The process can involve substantial document review and careful consideration of competing proposals.

I designed and developed an AI-assisted system to support this process by evaluating grant proposals against donor-specific requirements and producing structured assessment reports.

Rather than building a general-purpose AI chatbot, I approached the problem as a defined evaluation workflow: prepare donor knowledge, submit a proposal, retrieve relevant funding requirements, generate a structured assessment, calculate weighted scores and produce a report for human review.

My contribution covered problem definition, product design, evaluation criteria, technical architecture, implementation, integration and testing. The project was completed and successfully defended as an academic project, resulting in a functional prototype evaluated using simulated donor documents and benchmark proposals.

02

The problem

Evaluating grant proposals involves more than reading an application and deciding whether it deserves funding. Reviewers need to consider whether a proposal aligns with a donor's priorities, satisfies eligibility requirements, presents a feasible plan, justifies its budget and demonstrates the capacity to deliver its intended impact.

The project explored several challenges associated with this process:

  • —Reviewing large volumes of lengthy applications.
  • —Comparing proposals against multiple donor-specific requirements.
  • —Repetitive and resource-intensive assessment.
  • —Reviewer fatigue and cognitive overload.
  • —Differences in reviewer experience and judgment.
  • —Potential inconsistencies in scoring and recommendations.

These challenges informed the product's central objective: support reviewers with a structured, donor-aware assessment process while retaining human responsibility for funding decisions.

The problem areas formed part of the project's research and product framing. They were not established through a completed field study across donor organizations.

03

Product definition

The product was designed around a straightforward workflow:

Upload a proposalRetrieve donor requirementsGenerate an assessmentCalculate scoresReview and download a report

Intended users included grant reviewers, donor organizations, evaluation teams and funding decision-makers. The system was designed to provide an assessment that could be examined, shared and retained, rather than simply returning an unstructured AI-generated response.

04

My contribution

I was involved in both the product decisions and technical development of the system, including:

  • —Defining the problem, objectives and intended product workflow.
  • —Translating evaluation requirements into seven weighted criteria.
  • —Designing the donor knowledge preparation and proposal evaluation architecture.
  • —Selecting and integrating the document processing, retrieval and AI components.
  • —Developing the application workflow, interface and reporting capabilities.
  • —Separating AI-generated assessments from deterministic scoring logic.
  • —Establishing functional tests and benchmark comparisons.
  • —Investigating implementation and environment issues.
  • —Evaluating results and documenting product limitations.
05

Designing a decision-support product

One of the most important product decisions was positioning the application as a decision-support system rather than an autonomous grant approval engine. Grant funding decisions carry significant consequences, and an AI-generated recommendation should not be treated as a substitute for human judgment or accountability.

The system was therefore designed to generate structured assessments and recommendations that reviewers could examine before making funding decisions, shaping the product's output around:

  • —AI-generated criterion assessments.
  • —Strengths and weaknesses identified within proposals.
  • —Funding alignment analysis.
  • —Risk indicators.
  • —Weighted overall scores.
  • —Funding recommendations.
  • —A downloadable evaluation report.

The distinction between assistance and authority was central to the product concept.

06

Translating evaluation requirements into product logic

A useful evaluation system needs a repeatable framework; an unstructured AI opinion would not provide the consistency required by the product. I established seven weighted evaluation criteria to turn the evaluation concept into an explicit assessment framework:

Evaluation criterionWeight
Relevance25%
Feasibility20%
Expected impact20%
Sustainability15%
Budget justification10%
Organizational capacity5%
Innovation5%
Total100%

I also defined explicit recommendation thresholds:

Overall scoreRecommendation
90–100%Highly Recommended
80–89%Recommended
60–79%Needs Revision
Below 60%Not Recommended

An important design decision was separating AI interpretation from deterministic scoring. Google Gemini generated the criterion-level assessments and scores, while application logic calculated the weighted total and applied the recommendation thresholds—making the scoring rules explicit and avoiding relying on the language model to perform every calculation.

07

Grounding AI assessments in donor-specific knowledge

A conventional LLM implementation could submit a proposal to a model and ask it to evaluate the document, but general model knowledge alone would not reliably represent the specific requirements of an individual donor.

I introduced a Retrieval-Augmented Generation (RAG) architecture to address this. The system prepared donor reference material in advance, converted it into semantic embeddings and stored it in a vector database. When a proposal was submitted, the application retrieved relevant donor information and supplied that context alongside the proposal to Google Gemini.

The donor knowledge base included simulated documents covering:

  • —Funding guidelines.
  • —Eligibility requirements.
  • —Strategic priorities.
  • —Evaluation rubrics.
  • —Previously funded project summaries.

This allowed the evaluation workflow to draw on donor-specific reference material rather than relying exclusively on general model knowledge. The retrieval approach improved the structure of the evaluation process, although it did not guarantee that every relevant donor requirement would appear in every assessment.

08

Designing the system architecture

The architecture evolved during development. After implementing the donor document pipeline, I reviewed the original project structure and reorganised it around two distinct workflows so donor knowledge did not need to be extracted and indexed again whenever a reviewer submitted a new proposal:

Donor knowledge preparation

  1. 01Donor PDF documents
  2. 02Text extraction
  3. 03Text cleaning
  4. 04Semantic chunking
  5. 05Embedding generation
  6. 06ChromaDB vector storage

Proposal evaluation

  1. 01Proposal PDF upload
  2. 02Validation and text extraction
  3. 03Proposal embedding
  4. 04Donor context retrieval
  5. 05Gemini evaluation
  6. 06Weighted scoring
  7. 07PDF report generation

Separating preparation from evaluation meant the prepared knowledge base could be reused continuously by the online evaluation workflow. I also organised the application into modules with distinct responsibilities:

ModuleResponsibility
donor_pipelinePrepare and index donor documents
proposal_evaluationProcess proposals and generate assessments
uiProvide the user-facing interface
main.pyOrchestrate the evaluation workflow
testsVerify components and system behaviour
logsRecord processing and operational information

Within proposal_evaluation, separate responsibilities handled proposal processing, context retrieval, Gemini evaluation, scoring and report generation—simplifying the structure while preserving modularity.

09

Building the complete evaluation experience

The application integrated several independently developed components into one working workflow:

ComponentTechnologyPurpose
Core applicationPythonApplication logic and orchestration
PDF processingPyMuPDFExtract document text
Semantic embeddingsSentence TransformersRepresent documents semantically
Embedding modelmulti-qa-mpnet-base-dot-v1Generate document embeddings
Vector databaseChromaDBStore and retrieve donor knowledge
AI evaluationGoogle GeminiGenerate structured assessments
User interfaceGradioSupport proposal submission and result review
PDF reportingReportLabGenerate downloadable reports

The donor knowledge pipeline processed six simulated donor PDFs containing 98 pages and populated the vector database with approximately 605 embeddings.

The user journey was kept deliberately focused:

  1. 01Upload a proposal.
  2. 02Initiate evaluation.
  3. 03Review the generated assessment.
  4. 04Download the PDF report.

Each report included a proposal summary, criterion-level scores, strengths, weaknesses, funding alignment, risk flags and an overall recommendation—ensuring the system produced a usable evaluation artifact rather than limiting the experience to a conversational response.

10

Managing implementation challenges

Several challenges during development required revisiting assumptions, investigating dependencies and testing intermediate outputs:

Simplifying the architecture

The original project structure became unnecessarily complicated as implementation progressed. I reviewed the organisation and consolidated related evaluation responsibilities into a more cohesive module, while maintaining a separate donor knowledge preparation pipeline.

Resolving embedding model availability

When setting up the application on another machine, the embedding model could not load because local-only loading was enabled while the model was absent from that machine's cache. I adjusted the setup to allow downloading while internet access was available, then restored local-only loading once cached—highlighting the importance of reproducible environments and dependency management.

Investigating retrieval and prompt construction

I tested the retrieval workflow and inspected the resulting prompt rather than relying solely on the final model response. A recorded integration test retrieved 10 chunks and generated a prompt of approximately 24,786 characters, with the required prompt sections present—verifying intermediate pipeline stages while highlighting the need to consider retrieval coverage and long-document handling.

Integrating the complete workflow

Individual components did not automatically constitute a working application. Document extraction, retrieval, AI evaluation, scoring and reporting needed to exchange information in the correct sequence and format. Connecting these components through the main orchestrator with error handling and logging enabled the application to complete an end-to-end evaluation on the development machine.

11

Testing and evaluating the product

A working AI application is not necessarily a reliable one. I included functional testing and benchmark comparisons to examine whether the system behaved as expected and how closely its assessments aligned with predefined results.

Functional testing

Eight documented functional test cases were evaluated:

Test caseResult
Upload valid PDFPass
Reject invalid file formatPass
Extract proposal textPass
Generate proposal embeddingPass
Retrieve donor contextPass
Generate structured AI evaluationPass
Calculate evaluation scoresPass
Generate PDF reportPass

All eight documented functional test cases passed. This represents the outcome of the specified functional tests, not a claim of 100% AI accuracy.

Benchmark evaluation

I compared the system's generated scores against predefined benchmark scores for five simulated grant proposals:

ProposalBenchmarkAI scoreAbsolute difference
A98%95%3 pp
B63%57%6 pp
C20%20%0 pp
D63%58%5 pp
E98%99%1 pp

The mean absolute difference across the five proposals was 3 percentage points. This is a preliminary comparison against a small simulated benchmark dataset, not a general measure of real-world accuracy.

Recommendation agreement

I also compared the system's funding recommendations with the benchmark recommendations:

ProposalBenchmarkAI recommendationMatch
AFundFundYes
BReviseReviseYes
CRejectRejectYes
DReviseRejectNo
EFundFundYes

The system matched four out of five benchmark recommendations. One mismatch occurred on Proposal D, where the system recommended rejection instead of revision. Retaining this discrepancy in the evaluation rather than presenting results as perfect agreement highlighted the need for further testing and calibration before considering real-world funding workflows.

12

Product limitations and future opportunities

The completed application was a functional prototype, not a production-ready organizational platform. Its evaluation used simulated donor documents and a limited set of simulated proposals, leaving several capabilities requiring further development before deployment in an actual donor organization:

Identified limitationPotential improvement
Single simulated donor knowledge baseSupport multiple donor profiles and knowledge bases
Limited proposal benchmark datasetExpand evaluation coverage with authorised real-world data
Dependence on GeminiEvaluate alternative models and resilience strategies
PDF quality affects evaluationImprove document validation and extraction
Prototype Gradio interfaceDevelop a more complete production interface
No user authenticationIntroduce secure authentication
No proposal historyAdd evaluation history and record management
No multi-user managementIntroduce roles and access control

If development continued, I would prioritise evaluation reliability before expanding the feature set—focusing first on retrieval coverage, expanding the benchmark dataset, investigating recommendation mismatches and testing long-document handling before introducing multi-donor support, authentication and multi-user workflows.

Longer-term opportunities include proposal improvement suggestions, multilingual evaluation, model comparisons and enterprise deployment. These remain proposed improvements rather than features delivered in the completed project.

13

What this experience demonstrates

The project provided practical experience across product design, technical architecture and applied AI evaluation:

  • —Problem framing: Translating a broad operational challenge into a defined product workflow.
  • —Product design: Establishing intended users, evaluation criteria, outputs and user interactions.
  • —Requirements definition: Converting donor assessment needs into explicit scoring rules and recommendation thresholds.
  • —Technical product development: Translating product requirements into a modular AI-enabled architecture.
  • —AI product development: Integrating embeddings, semantic retrieval, vector storage and LLM evaluation.
  • —Workflow design: Connecting document processing, retrieval, scoring and reporting into one user journey.
  • —Quality assurance: Defining and executing functional tests.
  • —Product evaluation: Comparing generated results against predefined benchmarks and documenting discrepancies.
  • —Iterative problem-solving: Revisiting architectural decisions and resolving implementation constraints.
  • —Technical communication: Preparing project documentation, an eight-slide defense presentation and a 90-second product demonstration.
14

Reflection

This project reinforced that building a useful AI product involves considerably more than integrating a language model. The quality of the underlying information, retrieval process, business rules, application workflow and evaluation methodology all influence the usefulness of the final result.

One of the most important decisions was separating AI interpretation from deterministic scoring; another was retaining human oversight in a product connected to consequential funding decisions.

The project strengthened my understanding of how product requirements translate into technical architecture, how independently developed components come together into a complete user experience, and why testing must examine both functionality and output quality—taking an idea from problem definition through design, technical development and evaluation while recognising the difference between a working prototype and a product validated for real-world use.

Start a conversation

If you are navigating a product challenge, planning an initiative that needs structured delivery, or organising an event, I welcome thoughtful enquiries.

Open for conversations regarding

  • 01Product and project opportunities
  • 02Collaboration
  • 03Advisory and consulting engagements
  • 04Speaking and workshops
  • 05Other relevant professional enquiries