AI-Powered Grant Proposal Evaluation System
Designed and developed an AI-assisted decision-support prototype to evaluate grant proposals against donor-specific requirements, combining Retrieval-Augmented Generation, Google Gemini, deterministic weighted scoring, and downloadable PDF reports.
- Project
- Final-year Computer Science project
- Institution
- Usmanu Danfodiyo University, Sokoto
- Product type
- AI-assisted decision-support application
Case Study Contents (14)
Project overview
Grant proposal evaluation requires reviewers to examine lengthy applications against funding guidelines, eligibility requirements and multiple assessment criteria. The process can involve substantial document review and careful consideration of competing proposals.
I designed and developed an AI-assisted system to support this process by evaluating grant proposals against donor-specific requirements and producing structured assessment reports.
Rather than building a general-purpose AI chatbot, I approached the problem as a defined evaluation workflow: prepare donor knowledge, submit a proposal, retrieve relevant funding requirements, generate a structured assessment, calculate weighted scores and produce a report for human review.
My contribution covered problem definition, product design, evaluation criteria, technical architecture, implementation, integration and testing. The project was completed and successfully defended as an academic project, resulting in a functional prototype evaluated using simulated donor documents and benchmark proposals.
The problem
Evaluating grant proposals involves more than reading an application and deciding whether it deserves funding. Reviewers need to consider whether a proposal aligns with a donor's priorities, satisfies eligibility requirements, presents a feasible plan, justifies its budget and demonstrates the capacity to deliver its intended impact.
The project explored several challenges associated with this process:
- —Reviewing large volumes of lengthy applications.
- —Comparing proposals against multiple donor-specific requirements.
- —Repetitive and resource-intensive assessment.
- —Reviewer fatigue and cognitive overload.
- —Differences in reviewer experience and judgment.
- —Potential inconsistencies in scoring and recommendations.
These challenges informed the product's central objective: support reviewers with a structured, donor-aware assessment process while retaining human responsibility for funding decisions.
The problem areas formed part of the project's research and product framing. They were not established through a completed field study across donor organizations.
Product definition
The product was designed around a straightforward workflow:
Intended users included grant reviewers, donor organizations, evaluation teams and funding decision-makers. The system was designed to provide an assessment that could be examined, shared and retained, rather than simply returning an unstructured AI-generated response.
My contribution
I was involved in both the product decisions and technical development of the system, including:
- —Defining the problem, objectives and intended product workflow.
- —Translating evaluation requirements into seven weighted criteria.
- —Designing the donor knowledge preparation and proposal evaluation architecture.
- —Selecting and integrating the document processing, retrieval and AI components.
- —Developing the application workflow, interface and reporting capabilities.
- —Separating AI-generated assessments from deterministic scoring logic.
- —Establishing functional tests and benchmark comparisons.
- —Investigating implementation and environment issues.
- —Evaluating results and documenting product limitations.
Designing a decision-support product
One of the most important product decisions was positioning the application as a decision-support system rather than an autonomous grant approval engine. Grant funding decisions carry significant consequences, and an AI-generated recommendation should not be treated as a substitute for human judgment or accountability.
The system was therefore designed to generate structured assessments and recommendations that reviewers could examine before making funding decisions, shaping the product's output around:
- —AI-generated criterion assessments.
- —Strengths and weaknesses identified within proposals.
- —Funding alignment analysis.
- —Risk indicators.
- —Weighted overall scores.
- —Funding recommendations.
- —A downloadable evaluation report.
The distinction between assistance and authority was central to the product concept.
Translating evaluation requirements into product logic
A useful evaluation system needs a repeatable framework; an unstructured AI opinion would not provide the consistency required by the product. I established seven weighted evaluation criteria to turn the evaluation concept into an explicit assessment framework:
| Evaluation criterion | Weight |
|---|---|
| Relevance | 25% |
| Feasibility | 20% |
| Expected impact | 20% |
| Sustainability | 15% |
| Budget justification | 10% |
| Organizational capacity | 5% |
| Innovation | 5% |
| Total | 100% |
I also defined explicit recommendation thresholds:
| Overall score | Recommendation |
|---|---|
| 90–100% | Highly Recommended |
| 80–89% | Recommended |
| 60–79% | Needs Revision |
| Below 60% | Not Recommended |
An important design decision was separating AI interpretation from deterministic scoring. Google Gemini generated the criterion-level assessments and scores, while application logic calculated the weighted total and applied the recommendation thresholds—making the scoring rules explicit and avoiding relying on the language model to perform every calculation.
Grounding AI assessments in donor-specific knowledge
A conventional LLM implementation could submit a proposal to a model and ask it to evaluate the document, but general model knowledge alone would not reliably represent the specific requirements of an individual donor.
I introduced a Retrieval-Augmented Generation (RAG) architecture to address this. The system prepared donor reference material in advance, converted it into semantic embeddings and stored it in a vector database. When a proposal was submitted, the application retrieved relevant donor information and supplied that context alongside the proposal to Google Gemini.
The donor knowledge base included simulated documents covering:
- —Funding guidelines.
- —Eligibility requirements.
- —Strategic priorities.
- —Evaluation rubrics.
- —Previously funded project summaries.
This allowed the evaluation workflow to draw on donor-specific reference material rather than relying exclusively on general model knowledge. The retrieval approach improved the structure of the evaluation process, although it did not guarantee that every relevant donor requirement would appear in every assessment.
Designing the system architecture
The architecture evolved during development. After implementing the donor document pipeline, I reviewed the original project structure and reorganised it around two distinct workflows so donor knowledge did not need to be extracted and indexed again whenever a reviewer submitted a new proposal:
Donor knowledge preparation
- 01Donor PDF documents
- 02Text extraction
- 03Text cleaning
- 04Semantic chunking
- 05Embedding generation
- 06ChromaDB vector storage
Proposal evaluation
- 01Proposal PDF upload
- 02Validation and text extraction
- 03Proposal embedding
- 04Donor context retrieval
- 05Gemini evaluation
- 06Weighted scoring
- 07PDF report generation
Separating preparation from evaluation meant the prepared knowledge base could be reused continuously by the online evaluation workflow. I also organised the application into modules with distinct responsibilities:
| Module | Responsibility |
|---|---|
| donor_pipeline | Prepare and index donor documents |
| proposal_evaluation | Process proposals and generate assessments |
| ui | Provide the user-facing interface |
| main.py | Orchestrate the evaluation workflow |
| tests | Verify components and system behaviour |
| logs | Record processing and operational information |
Within proposal_evaluation, separate responsibilities handled proposal processing, context retrieval, Gemini evaluation, scoring and report generation—simplifying the structure while preserving modularity.
Building the complete evaluation experience
The application integrated several independently developed components into one working workflow:
| Component | Technology | Purpose |
|---|---|---|
| Core application | Python | Application logic and orchestration |
| PDF processing | PyMuPDF | Extract document text |
| Semantic embeddings | Sentence Transformers | Represent documents semantically |
| Embedding model | multi-qa-mpnet-base-dot-v1 | Generate document embeddings |
| Vector database | ChromaDB | Store and retrieve donor knowledge |
| AI evaluation | Google Gemini | Generate structured assessments |
| User interface | Gradio | Support proposal submission and result review |
| PDF reporting | ReportLab | Generate downloadable reports |
The donor knowledge pipeline processed six simulated donor PDFs containing 98 pages and populated the vector database with approximately 605 embeddings.
The user journey was kept deliberately focused:
- 01Upload a proposal.
- 02Initiate evaluation.
- 03Review the generated assessment.
- 04Download the PDF report.
Each report included a proposal summary, criterion-level scores, strengths, weaknesses, funding alignment, risk flags and an overall recommendation—ensuring the system produced a usable evaluation artifact rather than limiting the experience to a conversational response.
Managing implementation challenges
Several challenges during development required revisiting assumptions, investigating dependencies and testing intermediate outputs:
Simplifying the architecture
The original project structure became unnecessarily complicated as implementation progressed. I reviewed the organisation and consolidated related evaluation responsibilities into a more cohesive module, while maintaining a separate donor knowledge preparation pipeline.
Resolving embedding model availability
When setting up the application on another machine, the embedding model could not load because local-only loading was enabled while the model was absent from that machine's cache. I adjusted the setup to allow downloading while internet access was available, then restored local-only loading once cached—highlighting the importance of reproducible environments and dependency management.
Investigating retrieval and prompt construction
I tested the retrieval workflow and inspected the resulting prompt rather than relying solely on the final model response. A recorded integration test retrieved 10 chunks and generated a prompt of approximately 24,786 characters, with the required prompt sections present—verifying intermediate pipeline stages while highlighting the need to consider retrieval coverage and long-document handling.
Integrating the complete workflow
Individual components did not automatically constitute a working application. Document extraction, retrieval, AI evaluation, scoring and reporting needed to exchange information in the correct sequence and format. Connecting these components through the main orchestrator with error handling and logging enabled the application to complete an end-to-end evaluation on the development machine.
Testing and evaluating the product
A working AI application is not necessarily a reliable one. I included functional testing and benchmark comparisons to examine whether the system behaved as expected and how closely its assessments aligned with predefined results.
Functional testing
Eight documented functional test cases were evaluated:
| Test case | Result |
|---|---|
| Upload valid PDF | Pass |
| Reject invalid file format | Pass |
| Extract proposal text | Pass |
| Generate proposal embedding | Pass |
| Retrieve donor context | Pass |
| Generate structured AI evaluation | Pass |
| Calculate evaluation scores | Pass |
| Generate PDF report | Pass |
All eight documented functional test cases passed. This represents the outcome of the specified functional tests, not a claim of 100% AI accuracy.
Benchmark evaluation
I compared the system's generated scores against predefined benchmark scores for five simulated grant proposals:
| Proposal | Benchmark | AI score | Absolute difference |
|---|---|---|---|
| A | 98% | 95% | 3 pp |
| B | 63% | 57% | 6 pp |
| C | 20% | 20% | 0 pp |
| D | 63% | 58% | 5 pp |
| E | 98% | 99% | 1 pp |
The mean absolute difference across the five proposals was 3 percentage points. This is a preliminary comparison against a small simulated benchmark dataset, not a general measure of real-world accuracy.
Recommendation agreement
I also compared the system's funding recommendations with the benchmark recommendations:
| Proposal | Benchmark | AI recommendation | Match |
|---|---|---|---|
| A | Fund | Fund | Yes |
| B | Revise | Revise | Yes |
| C | Reject | Reject | Yes |
| D | Revise | Reject | No |
| E | Fund | Fund | Yes |
The system matched four out of five benchmark recommendations. One mismatch occurred on Proposal D, where the system recommended rejection instead of revision. Retaining this discrepancy in the evaluation rather than presenting results as perfect agreement highlighted the need for further testing and calibration before considering real-world funding workflows.
Product limitations and future opportunities
The completed application was a functional prototype, not a production-ready organizational platform. Its evaluation used simulated donor documents and a limited set of simulated proposals, leaving several capabilities requiring further development before deployment in an actual donor organization:
| Identified limitation | Potential improvement |
|---|---|
| Single simulated donor knowledge base | Support multiple donor profiles and knowledge bases |
| Limited proposal benchmark dataset | Expand evaluation coverage with authorised real-world data |
| Dependence on Gemini | Evaluate alternative models and resilience strategies |
| PDF quality affects evaluation | Improve document validation and extraction |
| Prototype Gradio interface | Develop a more complete production interface |
| No user authentication | Introduce secure authentication |
| No proposal history | Add evaluation history and record management |
| No multi-user management | Introduce roles and access control |
If development continued, I would prioritise evaluation reliability before expanding the feature set—focusing first on retrieval coverage, expanding the benchmark dataset, investigating recommendation mismatches and testing long-document handling before introducing multi-donor support, authentication and multi-user workflows.
Longer-term opportunities include proposal improvement suggestions, multilingual evaluation, model comparisons and enterprise deployment. These remain proposed improvements rather than features delivered in the completed project.
What this experience demonstrates
The project provided practical experience across product design, technical architecture and applied AI evaluation:
- —Problem framing: Translating a broad operational challenge into a defined product workflow.
- —Product design: Establishing intended users, evaluation criteria, outputs and user interactions.
- —Requirements definition: Converting donor assessment needs into explicit scoring rules and recommendation thresholds.
- —Technical product development: Translating product requirements into a modular AI-enabled architecture.
- —AI product development: Integrating embeddings, semantic retrieval, vector storage and LLM evaluation.
- —Workflow design: Connecting document processing, retrieval, scoring and reporting into one user journey.
- —Quality assurance: Defining and executing functional tests.
- —Product evaluation: Comparing generated results against predefined benchmarks and documenting discrepancies.
- —Iterative problem-solving: Revisiting architectural decisions and resolving implementation constraints.
- —Technical communication: Preparing project documentation, an eight-slide defense presentation and a 90-second product demonstration.
Reflection
This project reinforced that building a useful AI product involves considerably more than integrating a language model. The quality of the underlying information, retrieval process, business rules, application workflow and evaluation methodology all influence the usefulness of the final result.
One of the most important decisions was separating AI interpretation from deterministic scoring; another was retaining human oversight in a product connected to consequential funding decisions.
The project strengthened my understanding of how product requirements translate into technical architecture, how independently developed components come together into a complete user experience, and why testing must examine both functionality and output quality—taking an idea from problem definition through design, technical development and evaluation while recognising the difference between a working prototype and a product validated for real-world use.