Embedded: Innovation in Action Series
Complex urban challenges require thoughtful solutions. The Urban Innovation Fellows are working hard to deliver for New York City. The Embedded: Innovation in Action Series is a three-part series on our blog that will explore the innovative work of Cornell Tech’s Urban Innovation Fellows through the lens of three critical themes shaping the future of cities: Accelerating Decarbonization, Modernizing Public-Private Collaboration, and Leveraging AI and Data. Each post in the series will showcase Fellows’ work products, challenges, and successes to date, highlighting real-world projects developed in partnership with government agencies.
Imagine a drone flying low over the East River, capturing high-resolution imagery of seawalls, piers, bulkheads, and other maritime infrastructure that stretches for 520 miles along New York City’s waterfront. Beneath the water’s surface, remotely operated vehicles gather data on submerged assets that are difficult, expensive, and dangerous to inspect. Artificial intelligence analyzes these images, identifies early signs of deterioration, and helps prioritize maintenance before costly failures occur.
Only a few years ago, this would have sounded like science fiction. Today, it is the foundation of the New York City Economic Development Corporation’s (NYCEDC) innovative waterfront mapping initiative, a project looking to answer the question: What is the most effective and efficient way to capture, process, and prioritize waterfront imagery to enable streamlined inspections? To answer this question, rather than specifying a particular technological approach or vendor type as with a traditional procurement, NYCEDC challenged the market to propose creative solutions that could deliver above-water imagery, underwater inspection capabilities, and predictive maintenance tools.
Through an integrated pilot phase as part of this procurement, multiple vendors will demonstrate their technologies in real-world conditions before any decision about scaling or long-term procurement is made. This isn’t just a new way of thinking about waterfront inspection; it’s a new way for government to enable innovation.
Three Years Ago, This Wouldn’t Have Been Possible
Just three years ago, the options and ambitions of government technology pilots in NYC were limited. The fastest options, low- and no-cost pilots, were limited in size and had no path to scale. Demonstration Projects, the official procurement rule for piloting, allowed more expensive pilots but still left piloting agencies and vendors in the lurch after a successful pilot. Cornell Tech’s 2023 Pilot:NYC report, co-authored with NYCEDC, highlighted these issues, and proposed a solution: Challenge-Based Procurement.
Challenge-Based Procurement turns the traditional procurement process on its head. In a traditional solicitation, a government prescribes a solution with requirements, solicits proposals accordingly, evaluates those proposals primarily on written materials and past performance, and chooses the lowest-cost option. In a challenge solicitation, the government instead identifies a problem and the project’s goals. Vendors don’t just provide a quote, they propose a solution. The government agency can choose one or more options to pilot, and can then scale successful, validated solutions once the pilot is complete.
The New York City Mayor’s Office of Contract Services (MOCS) took on the task of making Challenge-Based Procurement a reality in City contracting. After examining federal, state, and local rules, MOCS saw that Challenge-Based Procurement was an achievable process change under the Demonstration Project Method, and that updating the City’s Procurement Policy Board (PPB) rules would enable greater impact. The necessary PPB revisions passed in 2024:
expanded the use of the Demonstration Project Method,
removed language limiting available procurement methods for future contracts,
extended the maximum term for contract extensions,
and formally established Challenge-Based Procurement in the City’s procurement rules.
By applying existing contracting structures, the PPB ensured that successful pilots could scale without requiring a new procurement or time-consuming contract amendments.
But simply establishing a new procurement approach isn’t enough. In order to take Challenge-Based Procurement from a novel and untested idea to a standard and accessible procurement tool, MOCS knew they needed to build a track record of success. MOCS sought out agencies who were willing to try something new and learn alongside the Procurement Operations team in real-time. MOCS partnered with early adopter agencies, including the Department of Citywide Administrative Services (DCAS), Department of Corrections (DOC), NYC Small Business Services (SBS), Department of Social Services (DSS), and the Mayor’s Office of Food Policy to identify opportunities to test and refine Challenge-Based Procurement across a range of procurement needs.
NYCEDC was eager to join the cohort of early-adopters. As a public benefit corporation, NYCEDC is not required to follow the same procurement rules as NYC Agencies—but was excited to partner with the Mayor’s Office of Contract Services to be aligned with the City and test out Challenge-Based Procurement. Procurement reform gets at the heart of NYCEDC’s priorities: growing business in NYC, not just through shaping economic policy, but as a customer. EDC is especially supportive of the opportunity that Challenge-Based Procurement offers to businesses, especially start-ups, who can prove product fit through a pilot phase, rather than relying solely on historical track record.
Piloting new technologies is already part of EDC’s portfolio of work through programs like Pilots at BAT, which advances technologies via short-term no-cost pilots at the Brooklyn Army Terminal, and Climate Innovation Studios, which supports catalytic projects with technical assistance, grants, and access to industry experts.
Despite these existing programs, until challenge-based procurement, there was no pathway from a successful pilot to larger procurement, a disservice to both the piloting company and to NYCEDC. This “pilot purgatory” was especially evident in Meera’s prior work at Newlab, where she partnered with the City on a number of pilot projects that led to meaningful insights and notable press moments, but didn’t translate directly into scaled-up procurement.
Finding the Right Challenge
A successful challenge begins with identifying the right problem, and works when the agency genuinely does not know the optimal solution and is willing to learn from the market.
Rather than searching for just any viable use case, NYCEDC staff spent significant time exploring potential challenges, evaluating whether various problems were truly right for a challenge-based model versus a traditional procurement.
The waterfront inspection challenge emerged because it met several important criteria: it was operationally critical, technologically complex, and lacked a clear best solution. Existing inspection methods are resource-intensive, and available waterfront imagery has not been comprehensively updated in many years. At the same time, advances in drones, robotics, underwater vehicles, photogrammetry, artificial intelligence, and predictive analytics suggested that the private sector might have solutions that the government had not yet considered.
Building a Market, Not Just an RFP
A challenge can only be successful if the right companies know about it and choose to participate. NYCEDC quickly realized that releasing a solicitation alone would not be enough.
Because the waterfront mapping challenge sat at the intersection of multiple technologies and use cases, it became apparent that many relevant companies may have never encountered NYCEDC through traditional procurement channels.
To address this, the team invested heavily in market-building activities before and during the solicitation process. The Robotics and Drones Partner Form was created to identify companies interested in working in New York City’s emerging robotics ecosystem and to build relationships before the procurement launched. The team conducted extensive outreach, engaged ecosystem organizations, amplified information through newsletters and industry networks, and held conversations with dozens of companies to better understand the state of the market.
The outreach strategy extended beyond digital marketing to information sessions and networking events, including a drones and robotics reception that brought together dozens of startups, established firms, City agencies, and ecosystem stakeholders. These gatherings helped companies identify potential partners and form teams capable of addressing different parts of the challenge.
The most successful City agency challenges, in terms of the number and variety of proposals received, used a multi-pronged approach to market-building. These agencies also gave their Challenges catchy, descriptive names, posted extensively among their professional networks, created webpages to drum up interest and provide an easy-to-access landing page ahead of the official solicitation release, and led site visits for prospective vendors.
Slotting a Challenge-Based Procurement into the traditional procurement process will limit its impact. In order for this innovative procurement approach to deliver the best results, innovative approaches are required for outreach and marketing, too.
Planning a Path to Scale
Critical to the promise of Challenge-Based Procurement is not just bringing innovative approaches in the door, but finding pathways to scale success and avoid the trap of “pilot purgatory,” where successful pilots lack viable procurement pathways.
Because NYCEDC has a more flexible set of procurement rules, their path to scale is relatively straightforward. After a pilot phase, NYCEDC plans to procure a successful solution through a sole-source procurement, allowing NYCEDC to continue to work with the best vendor.
For City agencies, while sole source procurements are an option, they can only be applied in very limited circumstances. With the adoption of Challenge-Based Procurement, MOCS knew it was important to create a scaling mechanism to continue testing innovations that wouldn’t require an additional procurement, which could add a year to piloting timelines. Instead, MOCS creatively applied an existing Master Services Agreement and Task Order contracting structure, typically used to keep a set of vendors “on call” for a specific type of work like road repairs.
For Challenge-Based Procurements, City agencies award each selected vendor their own Master Services Agreement, with the full possible scope of work and budget. For the initial pilot, the agency issues a Task Order for a smaller pilot and cost. If the pilot is successful, the agency can expand testing by issuing an additional Task Order for a larger pilot and higher cost. If the pilot is unsuccessful, the agency has no obligation to give the vendor more work under the Master Agreement.
Planning a pathway to scale from the start ensures that successful pilots have the runway to continue past the initial testing phase, saving both City agencies and vendors time and money, and providing continuity of service.
Changing Methods, Changing Attitudes
Perhaps the most important insight from the EDC solicitation is that innovation is not necessarily the creation of an entirely new technology. Instead, innovation often comes from utilizing existing technologies in new ways to solve problems.
Proposal evaluations repeatedly reinforced this lesson. The strongest concepts combined multiple inspection methods, data collection tools, and analytical platforms into integrated solutions rather than relying on a single breakthrough technology. The value emerged from how these pieces worked together.
Challenge-Based Procurement created space for those consortia to emerge. By inviting vendors to define solutions rather than forcing them into predetermined specifications, NYCEDC enabled a more creative dialogue between government needs and market capabilities.
Challenge-Based Procurements require an innovation mindset that must extend beyond solicitation, marketing, and outreach, into evaluation, selection, and program management as well. Traditional procurement evaluations are designed to assess how well a proposal meets a set of requirements, and managing vendors usually focuses on compliance and risk mitigation.
For Challenge-Based Procurement to deliver innovative results, evaluators must be given the ability to compare “apples and oranges” and assess the possibilities of different creative solutions to determine which are worthy of testing. Beyond the minutia of evaluation, this requires a mindset shift: being open to new ideas, new approaches and new vendors. In managing the contracts, agencies must continually evaluate vendor performance against the goals of the challenge, and identify those that are delivering real results.
Eight Tips for Cities and Agencies Designing a Challenge-Based Procurement
Start with an intractable problem. Choose a challenge that remains unsolved and where no obvious solution exists.
Be willing to learn from the market. Challenge-Based Procurement works best when agencies genuinely do not know the answer in advance. The goal is to test multiple approaches and discover what works, not to validate a predetermined solution.
Cultivate the market. To deliver the best results, innovative approaches are required for outreach and marketing.
Build organizational buy-in. One enthusiastic champion is not enough. Successful challenges require support from project managers, subject matter experts, procurement staff, and leadership.
A mindset shift is key. Shoehorning Challenge-Based Procurement methods into a traditional procurement workflow won’t yield innovative results. An innovation mindset must extend beyond the solicitation, marketing, and outreach, into evaluation, selection, and program management as well.
Ensure funding is available. Innovation often requires up front resources for testing and piloting. Vendors need confidence that successful participation can lead somewhere meaningful.
Define innovation broadly. Innovation is not necessarily the creation of an entirely new technology. Innovation is often the creative combination of existing tools rather than a brand-new invention.
Be willing to learn. The greatest value of Challenge-Based Procurement may be the solutions that agencies never would have thought to request on their own.