AI Systems Tackle Technical Documents
A developer shared a project demonstrating an AI system that can interact with technical documents containing charts and tables. The system uses a multimodal Retrieval-Augmented Generation (RAG) approach to process both text and visual data. This type of tool could streamline the review of complex grant applications and technical reports in research funding agencies.
- Multimodal RAG systems represent a significant advance over text-only versions by integrating and interpreting diverse data types such as text, images, and tables simultaneously. This allows them to understand the full context of complex documents, such as technical manuals or scientific papers, where diagrams and charts are crucial for comprehension. A key technology enabling this is Contrastive Language-Image Pre-training (CLIP), which helps the AI understand the relationship between words and images. - In the context of grant funding, AI tools are already being used to streamline the application process for both applicants and reviewers. For grant seekers, generative AI can assist in drafting proposals and structuring responses, while funders use AI to parse applications, flag errors, and analyze data, significantly reducing review times. Some grant management software now includes AI-assisted screening to automatically compare applications against program eligibility criteria. - Major US funding agencies like the National Institutes of Health (NIH) and the National Science Foundation (NSF) have established specific policies regarding AI. The NIH, for instance, will not consider applications significantly developed by AI, and both agencies prohibit reviewers from using public AI tools to analyze proposals due to confidentiality concerns. - The adoption of AI in European public administration faces several barriers, including a lack of quality data, ethical concerns, and the need for greater internal expertise and funding. A study of 26 national AI strategies in Europe revealed a strong focus on data initiatives and private sector collaboration, but limited initiatives to improve internal government capacity. - Several European local governments are successfully using AI to improve public services. For example, Kortrijk, Belgium, uses a multilingual virtual assistant to help citizens access services, and Nicosia, Cyprus, has a mobility assistance project for citizens with disabilities that emphasizes public engagement and trust-building. However, a technological gap is widening between large and small municipalities. - Implementing AI in the public sector involves significant challenges beyond the technology itself. These include navigating legacy IT systems, ensuring data privacy and security, complying with regulations like GDPR, and managing organizational change by redefining job roles and upskilling employees. - The EU AI Act is a key piece of legislation shaping the use of AI in the public sector. To comply, government agencies are establishing governance frameworks, such as Estonia's "AI sandbox," which mandates quarterly bias audits and public model cards to ensure transparency and fairness. Countries are taking different approaches to enforcement, with the Netherlands using a decentralized model of various supervisory authorities and Spain creating a new central authority. - While AI can increase efficiency, its use in grant writing has notable pitfalls. AI models have been known to generate fake citations, making it crucial for users to verify all sources. Furthermore, applications that are obviously AI-written without human oversight have been disqualified for providing non-relevant answers.
