Seven Transformations for More Equitable and Sustainable Cities

Image: WRI Ross Center for Sustainable Cities

Image: WRI Ross Center for Sustainable Cities

World Resources Report

Chapter 8

Data Collection Practices – Improving Local Data through Community Engagement

Credible, open local data creates an opportunity to ensure sound policies and investments, understand their impacts on vulnerable communities, and improve governance processes in cities. The lack of disaggregated local data that could help diagnose problems is a huge handicap for decision-makers. Cities should transform data collection practices to gather more granular local data with community participation. Good data need not be expensive to collect and can improve decision-making, regardless of the income level of the city.


8.1 What Must Change and Why

Local, spatially disaggregated data is lacking

Decision-makers may not understand the need for better local data on the availability and quality of urban services, infrastructure, jobs, education, health services, and other necessities. Failure to analyze granular information collected from different parts of the city can mask vast inequities. The socioeconomic indicators needed to track levels of access and quality are not consistently defined or measured, and they are rarely expressed spatially, making it harder to identify under-served locations within the city. Without spatially disaggregated data, cities rely either on nationally collected data, which are then downscaled to the city level based on population or other simple assumptions, or on citywide data, which do not capture or convey the heterogeneity across neighborhoods or disparities within them. Standard satellite-based methods of analysis cannot accurately characterize informal settlements or slums or the absence of services there, and censuses do not typically include such settlements. The majority of remote-sensing studies only document outward growth, ignoring inner-city redevelopment and increasing density.

Poor-quality or unavailable data hinders decision-making

Data gaps at city and neighborhood levels lead to misinformed, ineffective, or incomplete policy responses. Inadequate data make it impossible to diagnose the gaps in access to services across neighborhoods, the externalities these create, and the vulnerabilities of different populations. Importantly, it leads to an inefficient and possibly incorrect allocation of scarce resources. Missing data hampers policymakers’ ability to make crucial decisions amid political pressure and conflicting advice. In struggling and emerging cities, with limited resources for collecting data, citywide surveys are performed infrequently, and they typically lack the detail needed to draw meaningful inferences about vulnerable and under-served population groups (see Box 10). Often, data are not shared across public sector agencies or with other stakeholders because there are no mechanisms or platforms with basic standards for data quality or consistent metrics to allow wider access. Sometimes, city leaders hesitate to share or use data for political reasons.

Studies suggest that official statistics used by governments and international agencies understate the extent of urban poverty in most low- and middle-income nations.296 Similarly, global efforts to monitor access to particular services, such as water and sanitation, underestimate the deprivations that urban dwellers face because they neglect to assess whether the services are of adequate quality.297 Our research shows that data on informal housing, as well as widely used global indicators on access to water and sanitation, underestimate the extent of the urban water and sanitation crisis. This contributes to ineffective planning and management of these services. For example, the Joint Monitoring Programme (JMP) of the United Nations has created universal categories to measure, monitor, and compare progress on water access in connection with the SDGs. However, these categories fail to meaningfully consider water quality, regularity of supply, or affordability, and the data are not detailed or granular enough to identify urban populations at risk (see Box 10). Although the JMP reports that a growing proportion of the global population is gaining access to running water, the share of urban populations receiving piped water has actually fallen since 1990. In 2015, less than half of the urban population in many developing nations was receiving piped water on premises.298

Data on land-use regulations and planning processes across cities categorized as struggling and emerging often fail to track the type of urban growth that is occurring. Our research found that, when remote-sensing data is combined with urban demographic and economic indicators, many lower-income cities are experiencing significantly more outward growth than upward growth. That is, they are sprawling outward at their periphery rather than building more mid- to high-rise buildings within existing city margins.

Better data would provide insights on what drives this often-unmanaged outward growth. These growth patterns have significant implications for providing services across the city, as well as access to both jobs and services for under-served populations. Figure 25 compares levels of access to jobs, education, and health care in the Mexico City metropolitan area within 30 minutes or less on foot or by public transit. It shows the stark difference in access to these opportunities between people living in one of the wealthiest neighborhoods of Mexico City and those living in the poorest areas. Wealth, in this case, is measured by the government’s Urban Marginalization Index, which combines social, economic, demographic and access to services indicators.299

Figure 25 | Access to jobs, education, and health care varies between wealthy and poor neighborhoods in the Mexico City metropolitan area

Note: These maps measure access to services available within a 30-minute ride by public transport or walking combined with the government’s Urban Marginalization Index. Better access is displayed in green, with red representing less access, and dark maroon representing neighborhoods with no coverage. Access to jobs is measured by the total number of formal jobs in each neighborhood from Mexico’s Economic Census, and travel time was calculated using the street grid and public transit map.

Source: Brito et al., 2021.

Box 10 | Data challenges often hinder action or may lead to misinformed policies

To address the lack of comparable city-level data on water and sanitation access, we compiled data on 15 cities in the global South. The data illustrate the true costs for households of accessing water, dealing with sanitation, and pursuing coping strategies when public services do not exist. The study also found that available city-level data on water and sanitation can be inaccurate, misleading, or outdated.

For example, in Mumbai, India, the most recent publicly available statistics on municipal water coverage date from the last census conducted in 2011. It reported that 82 percent of households had access to piped water within their own or nearby premises. Several local experts believe these numbers overestimate access, especially considering that over 40 percent of the city is made up of informal settlements that typically lack formal water connections. Data from one centrally located informal settlement with over 2,000 households showed that not one of them was connected to the public piped network. Although it lies within the utility’s jurisdiction, the city considers this settlement an “undeclared” slum, where households are not eligible for water connections.a Most households relied on buying water from tanker trucks, which cost 50 times more than piped water. In many other cities in the global South, residents of informal settlements also are denied access to piped water due to the politics of land use and informality.

In addition to missing, outdated, and inaccurate data on basic coverage, cities contend with a lack of official data on access. They do not know if the water supply is dependable, affordable, or safe. Households that ostensibly have access to piped water may not always have water available in the pipes. In Bengaluru, India, the piped network is shown to cover at least 70 percent of the city, yet in many neighborhoods water is only available for three hours and for only three days of the week. Few cities capture information on the affordability and quality of water services, especially for lowest-income users. Low-income households in cities such as Windhoek, Namibia, and Dar es Salaam, Tanzania, spend 9 percent and 17 percent of their income, respectively, on piped water where this exists (to access the World Health Organization’s recommended minimum quantity of water of 50 liters per person per day).b Where piped water does not exist, they spend even more—12 percent and 38 percent, respectively—on communal water supply.

Our study also highlighted how official city-level data fail to capture the prevalence of unsafe sanitation practices. For example, we know that many households dig a trench and discharge untreated human waste directly into a nearby waterway or stormwater drainage channel. The field researchers gathered data for our study in Bengaluru; Caracas, Venezuela; Cochabamba, Bolivia; Colombo, Sri Lanka; Kampala, Uganda; Karachi, Pakistan; Lagos, Nigera; Nairobi, Kenya; and Santiago de Cali, Colombia. All of them report that these self-provisioned drains for untreated sewage exist, but they acknowledge that there is no reliable way to estimate the percentage of households in the city that use them. Dhaka, Bangladesh, has self-provisioned drains, but these are primarily used in the urban periphery. Maputo, Mozambique, and Mzuzu, Malawi, do not have self-provisioned drains because households mainly use on-site sanitation in the form of pit latrines. The waste from a majority of these is discharged untreated into farms or streams, flooded out, or buried with high potential to contaminate surface and groundwater sources.c Ironically, some self-provisioned drains function as open sewers and are likely included in sewer estimates, thus leading to an overestimation of how many households have sewer access.

Without local, disaggregated, spatial, and up-to-date data to capture everyday realities, policies and approaches to improve access will not be effective.

Notes: a. Anand, 2017: 87; b. WHO and WEDC, 2013; c. Satterthwaite et al., 2019.

Source: WRI, 2018.

Cities lack the capacity to manage, share, and use data

Cities may not be able to assemble the data they need without help. Collecting data on levels of access to different services across the city can be extremely challenging. This lack of data is often an obstacle to making improvements or holding governments accountable. Spatial data for access indicators are often missing and must be gathered, mapped, combined with other data, and shared across service-providing agencies to ensure more integrated and inclusive planning. Cities need standards for collecting, sharing, and using representative, good-quality data to inform decision-making. Without such standards, it is difficult to compare conditions across cities and neighborhoods within cities in a consistent way. This makes it hard to understand how exclusion from core services affects people’s quality of life. Gauging the true costs of the services gap, for whole cities, regions, and nations, is difficult as well.

8.2 Priority Actions

A. Use new technologies and partnerships for better data and more granular local insights

New technologies and partnerships now make it possible for cities to bring together the high-quality, disaggregated spatial and socioeconomic data they need to make better decisions, regardless of whether they are rich or poor. An explosion of new insights from various sources is revolutionizing data collection and presents a breakthrough opportunity for under-resourced cities in particular. Tools to do rapid community surveys and gather crowdsourced information, anonymized mobile phone records, electronic transactions, and satellite imagery can generate unprecedented amounts of precise, granular information. “Big data” is proliferating, on traffic flows, cell phone communications, Internet usage, and financial and other transactions. Some of this is available to the public. This data can now be collected cost-effectively, frequently, and at high resolutions of spatial detail and disaggregation across socioeconomic groups.300 Cities can seize this opportunity to utilize different data sources, methods, technologies, and types of data. They can identify activity patterns, job growth locations, and levels of service coverage. Yet while ensuring that the needs of low-income populations are measured, cities must also ensure the protection of privacy for all. Better data creates an opportunity for all cities to become better equipped to diagnose and solve problems.

Ensuring that the data are disaggregated by population group and neighborhoods can help cities understand prevailing inequities. It can help identify gaps in access to services, especially in vulnerable communities and locations. For instance, under SDI’s Know Your City initiative, community-gathered data from thousands of informal settlements across approximately 500 cities is being used to improve conditions and upgrade services (see Box 11). Based on evidence that “inclusive outcomes demand inclusive knowledge and action,” the initiative trains residents of informal settlements in surveying, enumeration, and mapping so that they can bring these data to the attention of decision-makers and lobby for improved infrastructure and services.301

The United Nations’ New Urban Agenda and SDGs recognize the importance of more accurate data. Both include commitments to improve the quality of actionable data for cities and development. The Million Neighborhoods Map, a new global tool designed to detect gaps in services in informal settlements, is valuable in helping to identify communities with limited access to street networks, which can be a good proxy for access to other services, such as power, water, sanitation, and other infrastructure (see example of Dar es Salaam, Tanzania, in Figure 26). Low-access neighborhoods are colored red, whereas those with high access to streets are colored blue. Zooming in on a city on the Million Neighborhoods Map quickly identifies areas most likely to be informal settlements that have sprung up with little planning and possibly little essential infrastructure.

Figure 26 | New technologies help measure the level of access to street networks in Dar es Salaam, Tanzania

Source: Mansueto Institute for Urban Innovation, Million Neighborhoods Map,

Satellite imagery combined with aerial photography using drones provides a low-cost way to tackle the otherwise expensive exercise of land mapping. Increasingly, cities in China, India, Rwanda, Tanzania, and some Latin American countries are using these technologies to monitor development patterns, complete land cadasters, enforce land-use regulations, and collect tax revenue.302 Detailed drone images combined with satellite imagery generated over time can help monitor growth patterns, service provision in informal settlements, and the condition of open spaces and environmentally vital areas. This type of data can be very useful to bring transparency in land records and transactions and enable spatial planning (see Transformation 6).

Combining advanced technologies such as satellite imaging and drone surveys with community-gathered data is helping shed light on conditions in informal settlements (see Box 11). Satellite imagery can detect where peri-urban and rural areas are being converted to urban land use—formally or informally—and whether new urban settlements have access to core services. With increasing use of artificial intelligence, satellite imagery is being used to “train” machine learning algorithms to provide citywide land-use maps, maps of vulnerable environmental sites, locations of slums, and more.303 This has the power to drastically improve planning and policymaking in cities where technical capacity is limited and where even basic land-use maps do not exist or are not updated. However, these data obtained through satellite imagery and artificial intelligence need to be matched with fieldwork in a sample of locations to understand error rates and be checked against and supplemented with information obtained from communities and from ground-level surveys.

Satellite imagery provides only a rough sense of where informal settlements exist. Their type and form (shacks, tin or thatched roofs, mud huts) vary so much that algorithms cannot yet make sense of them in any standardized way. This is why ground truthing through fieldwork is so important, and why satellite imagery must be combined with far more detailed drone surveys and/or data directly gathered by communities. These combined approaches provide powerful new insights on the under-served. For example, a recent seven-year study by researchers to map informal settlements in Bengaluru using satellite imagery and machine learning combined with ground truthing efforts recorded about 2,000 informal settlements in the city, whereas government records showed fewer than 600.304

Box 11 | Using geospatial data with community mapping helps plan neighborhood improvements

Whenever official data are out of date or not available, as is often the case for the informal sector, cities can explore new data sets provided by other stakeholders. For instance, organizations such as Slum/Shack Dwellers International (SDI) or the Asian Coalition for Housing Rights have produced systematic and aggregated data on informal settlements. Data sets such as those developed under the Know Your City campaign, SDI’s global initiative, can provide unique insights and complement existing official surveys. This geo-referenced community-collected survey data can provide information on tenure security, service density, housing quality, and other issues related to informal settlements where little empirical knowledge is currently available (Figures B11.1 and B11.2).

Figure B11.1 superimposes drawn imagery with community maps of all structures in the informal settlement of Westpoint in Monrovia, Liberia. The combination of high-resolution imagery and community data can improve the accuracy of maps and contributes useful information for planning that could not be obtained until recently. For instance, based on the maps and underlying data, we can provide metrics of total settlement population, population density estimates, and calculate the amount of built-up and open area per capita. Such metrics are necessary to understand current conditions and implement planning improvements. Figure B11.2 shows the location of different infrastructure services (water points, toilets, electricity transformers, streetlights, garbage disposal, etc.) as well as services of interest to the community (schools and religious institutions).

Figure B11.1 | In the informal settlement of Westpoint in Monrovia, Liberia, community-gathered data are used with drone imagery to assess gaps in access to services

Source: SDI, 2018; Kallergis, 2018.

Figure B11.2 | A neighborhood map can pinpoint key services

Source: SDI, 2018; Kallergis, 2018.

In practice, community members, especially women and youth, are trained in digital data collection, including the double-checking of entered data, as a core practice. The information gathered is geo-referenced and then provides spatial data for settlement borders; service points such as water taps, toilet facilities, and health clinics; and major identifying landmarks nearby. All profiles automatically require a time stamp, physical contours, and surveyors’ names and contact information so information can be verified, assessed over time, and understood by third parties. Beyond producing geographic information, settlement profiling culminates with identifying risks and priorities for each community and careful enumerating of what can be broadly described as the social fabric of settlements (i.e., the presence of community leadership, the number of community-based organizations, savings groups, women and youth clubs, schools, churches, etc.).

B. Increase city capacity to collect and effectively utilize data

Credible spatial data disaggregated to identify vulnerable locations and communities and the ability to share such data publicly are key to creating a shared understanding of a problem. National and regional governments can help cities acquire the tools and build the capacity to gather, analyze, and share data while protecting people’s privacy. National and regional governments must invest in building technical capacity in cities and setting standards for consistent data collection, sharing, and use. Sharing data across government agencies within cities and between local, regional, and national agencies can facilitate greater transparency in decision-making and support citizen innovation and engagement. This is especially important if cities are to play a key role in solving problems that are regional in scope, such as curbing air pollution, managing water resources, mitigating climate risks, and conserving biodiversity and green spaces. For example, Figure 27 shows how spatial data can usefully highlight areas in Egyptian and Bangladeshi cities to prioritize for adaptation measures against sea level rise.

Figure 27 | Coastal flooding risks from sea level rise may impact large swaths of built-up areas including in Egypt and Bangladesh

Note: “Urban centers” are cities and large urban areas; “urban clusters” are towns and suburbs or small urban areas.

Source: Center for International Earth Science Information Network (Columbia University), CUNY Institute for Demographic Research (City University of New York), and the Institute of Development Studies, 2019. For the Coalition for Urban Transitions and the Global Commission on Adaptation. Published in Chu et al., 2019.

New technologies coupled with adequate resources and data sharing can give governments and planners unprecedented insight into a wide range of problems and potential solutions. For example, given the growing use of digitized maps to plan major infrastructure projects, it is possible to overlay data on core infrastructure networks (transport, water, sanitation, electricity) with settlement data, allowing stakeholders to analyze the need for future investments to expand service provision. It is also possible to further overlay these service networks with data on job locations, education, health care, transport hubs, and green spaces to identify key challenges facing marginalized populations across the city. Investing in city capacity to develop better data systems can be an extraordinarily productive way to spur innovation and improve decision-making, if city leaders are willing to attack the problems exposed through better data.

C. Coproduce and share data to foster more effective and inclusive governance

Investing in better data has many payoffs over time. It can enhance decision-making, improve quality of life for all sections of the population, increase efficiencies in resource use, help plan for and avoid costs of future risks, and support citizen participation in policymaking processes. Broader participation can improve local and regional governance and support reforms promoting economic development, education and training, public health, economic and social welfare, and resilience. Data collection and monitoring needs to be a continuous capability that governments invest in over time because data are constantly changing. Data sources and outputs must be compatible with each other over time to produce meaningful insights. Keeping up to date data requires partnerships across key data providers, communities, the private sector, local researchers, and universities that bring the data they have access to and help sustain this capability (see Figure 28).

Figure 28 | Data from multiple actors can lead to more effective and inclusive decision-making

Source: Authors.

Democratizing data production and access and integrating useful community knowledge can make governance and planning more inclusive. Beyond generating useful data, shifting to new paradigms of knowledge coproduction can lead to more inclusive outcomes for cities. Engaging community actors to produce and share data can both shed light on inequities and gaps in service and help craft policies to address them. But for this to work, data proliferation is not enough; the data must also be operationally useful. It needs to be organized, analyzed, and shared in ways that can help inform and mobilize support for better regulations and policies. The increasing availability of open data and common standards for data collection are making it easier to share crucial local-level information with multiple actors. Making such data more transparent can help energize citizen pressure and spark innovation for more sustainable growth. Self-enumeration projects conducted by SDI, NGOs, and community groups in Nairobi mapped over 50,000 households in the city, with community organizations collecting, producing, and using data to arrive at a common vision and set of priorities.305 These groups were then able to lobby the city’s water and sewer company to provide convenient water sources. In some of the largest informal settlements, such as Huruma, Kibera, Mathare, and Mukuru, residents were able to successfully challenge the city’s evictions and slum-clearance efforts and negotiate upgrading schemes with landlords.306

Voices: Rohit Aggarwala on data, governance and inequality

In addition to helping communities and citizens to identify and meet needs, data can help them uncover problems. For example, they can monitor development that is illegal and environmentally detrimental, built by private developers without government approvals. Examples include huge gated communities that sprang up in a lakebed on the periphery of Bengaluru. Environmental groups and future residents who invested in the community raised the alarm when they found out it had been built without environmental clearances in a sensitive ecological zone.307 Authorities may know illegal development is happening and ignore it. But making the data visible allows citizens to take action. The right information provides impetus for public action and allows coalitions of stakeholder groups to come together to make decisions around investment, set common objectives, negotiate mutual commitments, and lobby policymakers (see Figure 28). It enables them to monitor, track, and expose the impacts of public policies and private actions on equity and environmental sustainability and to hold decision-makers accountable. Data transparency and sharing can be politically empowering.

Data transparency and mandates to share data across jurisdictions and stakeholder groups can also help local authorities build trust and improve their performance. The Towards a More Equal City case study of Johannesburg describes the development of a comprehensive, publicly available database of landownership records in partnership with private developers and property owners. 308 The database proved valuable for the planning of an urban development zone meant to encourage economic growth and affordable housing in inner-city areas, along with the design of incentives for private developers.

Concerns have arisen in recent years around data privacy, private technology companies exploiting and selling data for profit, and taking the lead in driving the growth of “smart city” initiatives.309 Technology companies are advancing into delivering urban services such as transport (e.g., Uber, Ola, EasyTaxi), managing energy and water infrastructure, and deploying sensors to measure air and water quality, among other smart systems within cities. This can leave public sector agencies with low capacity, scrambling to manage this transition and protect citizens. Some cities have begun setting conditions for the collection, use, and sharing of data generated about citizen needs and choices. For example, São Paulo passed an innovative regulation requiring all transport network companies operating in the city to provide the city with data on trip origins, destinations, times, distances, travel routes, prices charged, and service evaluation by the customer. The data remain confidential and secure and provide invaluable real-time information that the city can use to plan transport services and operate the road network more efficiently, as well as to charge fees from these companies for the use of public streets.310 Although São Paulo has established a collaborative lab, MobiLab, involving transport professionals and computer and data scientists to analyze the data, many cities struggle with the technical capacity needed to draw insights from similar big data sources.

Relying on data generated by private companies may present other challenges. Data gathered from people’s smartphones as they move around the city typically miss some of the most vulnerable, who do not own or use a smartphone. Phone data then does not capture the behavior and preferences of those most at risk and are unable to provide any insights on questions of equity. It is important to note that data gathered from any source, even highly accurate data, can be used for a variety of purposes. Some governments have used new mapping and technology to clear slums, further harming marginalized groups rather than looking for ways to help them. It is important that these technologies should be used to include—not to exclude—the under-served. They should support enabling regulations to improve their living and working conditions, and they should be used to promote data transparency.

Our research shows that combining the information provided by a wide set of stakeholders—community groups, universities, NGOs, and the private sector—can help ensure its credibility and bring new insights. Gathering, sharing, and using better data can drive transformative change. It creates an enormous opportunity to respond more effectively to many of the practical problems that cities and neighborhoods face today. Table 4 lists the actions and roles required of different actors to move Transformation 3 forward.

Table 4 | Roles of specific actors in advancing Transformation 3: Data Collection Practices

Data Collection Practices—Improving Local Data through Community Engagement

City Government and Urban Sector Specialists

  • Build capacity to bring together high-quality, disaggregated spatial and socioeconomic data by harnessing new technologies and forging new partnerships
  • Utilize advanced technologies such as satellite imaging and drone surveys, supplemented with community-gathered data to understand access gaps and inform decision-making
  • Engage community actors in producing and sharing data, and integrate community knowledge in planning and governance processes
  • Democratize data access to support citizen participation in policymaking, build trust, and diagnose gaps in government action

National Government

  • Invest in building technical capacity in cities and set standards for consistent data collection, sharing, and usage
  • Develop guidance on urban metrics and evaluation indicators that can be used by regional and local agencies for inclusive planning
  • Invest in building inclusive governance practices informed by data gathered from vulnerable communities, such as by engaging civil services staff to enforce rules on transparency under good budgeting, accounting, and reporting standards and with community input

Civil Society, including Nongovernmental Organizations, Experts, and Researchers

  • Support communities to gather data on the quality and quantity of core services in informal settlements and neighborhoods where vulnerable groups reside
  • Advocate for data transparency and mandates to share data across jurisdictions and stakeholder groups to expose gaps in government action and improve governance processes
  • Influence decision-makers to shift to new paradigms of local planning and decision-making, based on coproducing and sharing knowledge across local community groups and a wider set of stakeholders, including universities and the private sector
  • Collaborate with public sector officials to help build technical capacity, train staff, and develop new approaches for collecting and using data to inform decision-making

Private Sector

  • Provide funds and expertise for cocreation of knowledge and collection of good-quality, consistent data to inform decision-making
  • Increase the availability of open data, new technologies to gather data, and the development of common standards for data collection
  • Partner with government bodies to access and share disaggregated data to enable problem-solving

International Community, including Development Finance Institutions

  • Create knowledge and develop tools and standards to make spatial, disaggregated, socioeconomic data available at local, national, and global levels
  • Incentivize data transparency and mandates to develop, maintain, and share data across jurisdictions and stakeholder groups in urban areas
  • Use peer exchange and knowledge networks to share lessons from good practices in designing integrated urban actions and policies that help achieve multiple desired goals, such as the Sustainable Development Goals and climate action at local, regional, and global levels
  • Influence decision-makers to shift to new paradigms of planning, governance, and operations, based on coproducing and sharing knowledge across community groups and a wider set of stakeholders, including universities and the global private sector
  • Provide financing to build technical capacity in cities to use data for decision-making and to set standards for consistent data collection, sharing, and usage

Source: Authors.