Special issue for Prof YL XU

Advances in Structural Engineering

2022, Vol. 0(0) 1–25

© The Author(s) 2022

Article reuse guidelines:

sagepub.com/journals-permissions

DOI: 10.1177/13694332221119883

journals.sagepub.com/home/ase

Performance-based post-earthquake

building evaluations using computer

vision-derived damage observations

Nathaniel M Levine1, Yasutaka Narazaki2 and Billie F Spencer, Jr1

Abstract

After a major earthquake, rapid community recovery is conditional on ensuring buildings are safe to reoccupy. Prior studies

have developed statistical and machine learning-based classifiers to characterize a building’s collapse capacity to resist an

aftershock given mainshock responses of the building. However, for rapid safety assessment, such a method must be

coupled with an automated inspection methodology to collect damage information. Furthermore, probabilistic models of

expected building performance must be updated based on the distribution of observed damage. This paper presents a

method for rapidly assessing the safety of a building by incorporating damage that has been identified and localized using

unmanned aerial vehicle images of the building. Probabilistic models of earthquake demands on exterior components are

directly updated using observed damage and Bayes’ Theorem. Updated demand models on interior components are then

inferred using a machine learning-based surrogate for the analysis model. Both sets of updated models are used to

determine if the building is safe to occupy. Results show that predictions of building demands are improved when

considering the observed damage. When combined with automated image collection and processing, the proposed

methodology will enable rapid, automated safety assessment of earthquake-affected buildings.

Keywords

earthquake engineering, structural health monitoring, machine learning, computer vision

Introduction

After a large earthquake, a rapid recovery is conditional on

people’s ability to quickly return their homes with minimal

disruptions to services (SPUR, 2012). As part of its Resilient City initiative, the San Francisco Bay Area Planning

and Urban Research Association (SPUR) defines a shelterin-place performance target, where residences are sufficiently safe to still provide adequate shelter and disruptions

to essential services, like roads, water, and power, are

minimal. Part of this shelter-in-place strategy relies on

quickly ensuring that buildings are safe to reoccupy in the

aftermath of an earthquake. Ensuring buildings are safe

requires a systematic and efficient way of assessing the postearthquake state of buildings in the impacted region. In the

United States, the ATC-20 standard (Applied Technology

Council, 1989, 1995) establishes a set of guidelines for field

inspections of earthquake-affected buildings.

Nevertheless, inspectors are limited in their investigations; simple destructive measures, such as removing architectural finishes to inspect the underlying structure, are

not typically performed. Instead, inspectors must infer

structural condition from nonstructural damage, based on

their own experience and judgment. After a detailed

evaluation, if the condition is still uncertain, the owner

must retain a consulting structural engineer to conduct an

engineering evaluation, involving detailed analysis and

destructive investigations. Overall, this inspection process

can be slow, both in terms of startup time to mobilize

inspectors and the time to physically visit and inspect

impacted buildings (Chock, 2007). The process is also

subjective; decisions are made based on the judgment

of inspectors working in a post-disaster environment

1

Department of Civil and Environmental Engineering, University of Illinois

at Urbana-Champaign, Urbana, IL, USA

2

Zhejiang University/University of Illinois at Urbana-Champaign Institute,

Zhejiang University, China

Corresponding author:

Nathaniel Levine, University of Illinois at Urbana-Champaign Newmark

Civil Engineering Laboratory, 205 N. Mathews Ave, Urbana, IL 61801,

USA.

Email: nlevine3@illinois.edu

(Burton and Deierlein, 2018), who generally do not have

access to centralized repositories of building drawings and

design information (Earthquake Engineering Research

Institute, 2011).

Assessing buildings in advance of an earthquake can

reduce potential post-earthquake downtime. For example,

as part of its Resilient City initiative, SPUR recommends

that housing be evaluated for its potential shelter-in-place

capacity in the aftermath of an earthquake (SPUR, 2012).

Several cities in California recognize this advantage, allowing building owners to retain licensed engineers to

evaluate their building and develop an inspection program,

which the city pre-approves (City and County of San

Francisco 2020; City of Berkeley 2021; Lorenz 2014).

For example, in San Francisco’s Building Occupancy

Resumption Program, the hired engineer will perform an

inspection and classify the building within hours after an

earthquake, rather than days or weeks.

When assessing a building in advance of an earthquake,

post-earthquake tagging can be considered as a target

performance objective. For example, Mitrani-Reiser

(2007) and Mitrani-Reiser et al. (2016) proposed the

Virtual Inspector to estimate the probability of a building

receiving a red, yellow or green tag using the Pacific

Earthquake Engineering Research Center (PEER)

performance-based earthquake engineering (PBEE)

framework (Porter, 2003). A nonlinear structural analysis is

used to estimate the probability of damage on structural

components, based on a set of component fragility curves

selected for consistency with damage descriptions in ATC20. Component fragility curves define the probability of

meeting or exceeding a discrete, qualitative damage state as

a function of an input engineering demand parameter

(EDP), such as interstory drift or floor accelerations.

Components can take on four damage states: no damage,

light, moderate and severe. Responses from a suite of

nonlinear time history analyses were used to construct a

probabilistic model of EDPs, given an input earthquake

intensity, which in turn is used to estimate the probability of

components being in a particular damage state. The predicted damage states are then used to estimate the probability of a building receive a red, yellow, or green tag for a

given earthquake intensity.

Subsequent work has extended the Virtual Inspector to

explicitly link visual damage indicators with reductions in a

damaged building’s collapse capacity (Burton and

Deierlein, 2018; Burton et al., 2017; Zhang and Burton,

2019; Zhang et al., 2018). This approach ties observable

damage to building performance in an aftershock, eliminating some of the subjectivity associated with current

post-earthquake inspection methods. A building is classified as unsafe if the aftershock collapse capacity, relative to

the pre-earthquake collapse capacity, drops below a predetermined threshold. For example, Burton and Deierlein

(2018) considered relative reduction in collapse capacity as

a function of component damage state ratio (DSR). The

DSR is the proportion of a particular component type in a

particular damage state. They calculated a triggering DSR,

the DSR that would result in an unacceptable reduction in

collapse capacity. Similar studies (Burton et al., 2017;

Zhang and Burton, 2019; Zhang et al., 2018) used EDPs

and associated component damage as input to various

classification and regression models to predict postearthquake building safety.

While the Virtual Inspector and its extensions offer a

systematic way to predict how a building is likely to

perform during an earthquake, they are limited in that they

are not meant to be used to tag a particular building in the

field. The main uses of the Virtual Inspector, for example,

are for pre-earthquake risk assessment, to provide rapid

safety alerts if a building has a high likelihood of damage,

and for post-event decision support for stakeholders

(Mitrani-Reiser et al., 2016). Similarly, Burton and

Deierlein (2018) developed methods for calibrating

building tagging criteria to meet specific performance

targets for minimizing post-earthquake collapse risk, rather

than to go into the field and classify a specific building.

Nevertheless, the analysis procedures developed in previous studies can be used to estimate a prior model of

expected seismic performance, which can be updated based

on observed damage.

Several studies have proposed using observed damage

to update probabilistic models of earthquake demands to

better estimate global damage and predict future performance. Yazgan and Dazio (2012) estimated a probability

distribution of the maximum drift a building would experience during an earthquake. Bayesian inference was

used to update the prior estimates for maximum drift given

observations of damage states and residual drift. The

posterior distribution was used to predict the properties and

performance of the damaged structure. Similarly, Reuland

et al. (2019) used observed damage and ambient vibrationbased period measurements to select candidate analysis

model parameters from a suite of randomly generated

models. However, these studies use simplified test specimens which are not representative of in-service buildings.

They also rely on manual collection of damage observations; automated inspection methodologies are necessary to

reduce the risk to human life.

Computer vision-based technologies have been proposed for automating building inspections, reducing the

need for human inspectors to physically visit a site. Such

systems typically rely on unmanned aerial vehicles (UAVs)

to collect images of a structure, and then use computer

vision, including image processing and machine learning,

to identify damage in photographs (Koch et al., 2014;

Morgenthal and Hallermann, 2014; Spencer et al., 2019).

Recently, convolutional neural networks have been

2 Advances in Structural Engineering 0(0)

employed to automatically detect various types of structural damage, including cracking, concrete spalling, exposed rebar, and steel corrosion, and to identify structural

components like beams and columns (Hoskere et al., 2017,

2018, 2020, 2022; Hüthwohl et al., 2019; Narazaki et al.,

2020, 2021; Yeum et al., 2018; Wang et al., 2020, 2022; Xu

et al., 2019). Researchers have correlated visual damage

with expected component damage progressions to automatically estimate maximum column drift demands (Paal

et al., 2015) and classify columns into fragility-consistent

damage states (Pan and Yang, 2020) based on photographs

of columns. However, to incorporate this information into a

global performance assessment, the damage needs be localized to specific building components, with known design and function within the structural system.

To this end, Levine and Spencer (2022) and Levine et al.

(in review) proposed a building information model (BIM)-

based digital twin that assign qualitative, fragilityconsistent damage states to components imaged by a

UAV survey. The work leverages previous work in construction monitoring that fused semantic information from

a BIM with photographs of a construction site to track

progress and compliance with the design. The BIM-based

digital twin starts with a photographic UAV survey of the

building, which is registered in the BIM reference frame.

Damage is automatically identified in the survey images

and in 3D point clouds generated from those images.

Subsequently, this damage is localized to specific BIM

components, which are then classified into damage states

consistent with component fragility curves. To decide

whether the building is safe to occupy, however, this information must be incorporated into a performance-based

earthquake assessment of the building to classify its postearthquake safety.

This paper proposes a methodology to incorporate

computer-vision derived estimates of building component

damage states into a performance-based earthquake assessment to enable rapid classification of the postearthquake safety state of the building. Compared to traditional inspection methods, the method proposed in this

paper enables a complete and rapid automated inspection

process, from data collection to final safety classification.

The methodology begins with a description of damage

states on each exterior building component, such as would

be obtained from a UAV survey and an application of the

BIM-based digital twin proposed by Levine and Spencer

(2022). First, a suite of nonlinear time history (NLTH)

analyses is used to develop a probabilistic model of

component demands conditioned on earthquake intensity.

For exterior components, these models are updated based

on the observed damage states using Bayes’ theorem and

appropriate component fragility curves. Next, a machine

learning-based surrogate model is trained to characterize

maximum earthquake demands on interior, nonvisible

components. The result is a set of updated probability

models that are used to predict the probability of exceeding

a safety limit state for the building. The full methodology is

demonstrated on an example building from the University

of Illinois Campus in Urbana, Illinois. A major contribution

of this paper is the demonstration of the methodology on an

in-service, full-scale building with irregularities, rather

than a simplified 3D model. Furthermore, the updated

distributions can be incorporated into an existing method

like the Virtual Inspector to condition tagging probabilities

on observed exterior damage; this integration would enable

an expedited detailed ATC-20 inspection of the target

building. When combined with an automated data collection and damage identification and localization strategy,

the proposed methodology will allow for rapid classification of building safety, supporting community recovery

after an earthquake.

Methodology

This section details the methodology for the proposed

framework. First, a summary of the framework is presented. Then, a brief overview of PBEE is provided, with

an emphasis on the assumptions and methods used in the

Virtual Inspector (Mitrani-Reiser et al., 2016) and incorporated into this paper. The procedures for creating a prior

model of component demands, Bayesian updating for

exterior components, and surrogate modeling to predict

interior demands are detailed.

Overview of the methodology

The goal of the methodology is to classify a damaged

building’s safety by updating estimates of maximum

earthquake demands on a building based on observed

exterior damage. An overview of the method is presented,

going from qualitative damage descriptions on the building

exterior to decisions on structural safety. A summary of the

framework is shown in Figure 1.

The methodology presented in this paper extends the

BIM-based digital twin framework proposed by Levine and

Spencer (2022) and Levine et al. (in review). Levine et al.

(in review) flew a photographic UAV survey of the target

structure and applied computer vision technologies to localize automatically detected damage to specific BIM

components. Each component was classified into a damage

state consistent with component fragility models, enabling

integration with a PBEE framework.

In this paper, the damage states estimated from UAV

observations are used to update predictions of maximum

earthquake demands on building components to classify

the building as safe or unsafe. An initial model of building

performance is developed to characterize how the building

is expected to behave in an earthquake. From a suite of

Levine et al. 3

NLTH analyses, the responses of each component are used

to fit a lognormal distribution for EDPs, conditioned on

input earthquake intensity. These distributions comprise a

prior model that is updated based on observations of

damage on exterior components. First, the prior models for

exterior components are directly updated based on the

damage state for each component. For each component, a

component fragility curve, or set of fragility curves, is

identified to characterize the relationship between EDPs

and visible damage; these fragility models are used to

estimate the probability of observing damage on a given

component, given an input earthquake intensity. Bayes

theorem is then applied to update the EDP probabilities for

exterior visible building components.

To characterize the conditional distributions on interior,

nonvisible components, a surrogate model is developed to

predict interior EDPs based on exterior EDPs. This surrogate model replaces the computationally expensive

nonlinear structural analysis model and consists of two

parts. First, a support vector machine classifies whether

post-elastic behavior has initiated on the interior component. Second, if post-elastic behavior has occurred, a neural

network predicts the EDP value. Correlated, random realizations of EDPs on the exterior components are generated and propagated through the surrogate to characterize

the distribution of EDPs on interior components, conditioned on external damage observations. The result is an

updated probability model for both exterior and interior

building components, conditioned on the results of the

UAV survey and the input earthquake intensity. These

updated probabilities for component EDPs are then compared against target performance objectives to estimate the

probability that the building is unsafe for occupation. The

next section introduces the background principles from

PBEE that are used throughout this paper to develop this

proposed framework.

Performance-based approach to post-earthquake

inspection

In PBEE, buildings are designed to meet system-wide

performance objectives, determined by the building

stakeholders. Example objectives can be to minimize the

risk of collapse, fatalities, repair costs, or downtime given

an input earthquake intensity (Goulet et al., 2007; MitraniReiser, 2007; Moehle and Deierlein, 2004; Porter, 2003). In

the Virtual Inspector proposed by Mitrani-Reiser et al.

(2016), the tagging probability is estimated from the

probability of DMij, the damage measure j on component

Figure 1. Summary of the proposed methodology.

4 Advances in Structural Engineering 0(0)

type i, given the corresponding EDP for component type i,

EDPi

P

DMij



NC, IM

¼

Z

P

DMij



EDPi



pðEDPjNC, IMÞdEDPi

(1)

Here, P[X|Y ] is the conditional probability of X given Y,

p(X|Y ) indicates the conditional probability density function (PDF) of X given Y. NC is the non-collapse condition

and IM is an earthquake intensity measure, such as spectral

acceleration. The conditional PDF p(EDPi|NC, IM) is

chosen as a lognormal distribution whose two parameters,

median and dispersion, are fit to results from a suite of

nonlinear time history analyses using the procedures developed by Miranda and Aslani (2003). The relationship

between DM and EDP is described by component fragility

curves.

Component fragility curves describe the probability of

meeting or exceeding a discrete damage state given an

EDP. The component fragilities used in the PEER methodology are a systematic tool for translating demands from

a structural analysis model to damage that would be observed by an inspector in the field. By doing so, they make

it possible to consider performance objectives like repair

cost, which are based on the severity of physical damage

rather than demands from a model. Typically, fragility

curves are parametrized as lognormal cumulative distribution functions (CDFs) and their parameters are determined through a combination of experimental testing,

analysis, and expert opinion (Applied Technology Council,

2018a). FEMA P-58 (Applied Technology Council, 2018a,

2018b) describes methods for calculating component fragilities and provides a tabulated reference of fragility curve

parameters for common structural and nonstructural

components (Applied Technology Council, 2018c). This

relationship between damage and EDP is used in this paper

to update the probabilities of EDPs based on observed

damage; these updated probabilities are then used to estimate building earthquake performance. The following

sections describe the methodology for developing a prior

probability model of expected EDPs and updating that

model based on observed external damage.

Prior model of seismic demands

The first step in developing the prior model of seismic

demands is to create a finite element model of the target

building. Nonlinearity is accounted for using concentrated

plastic hinges at the ends of beam and column elements.

Hinges are defined using cyclic backbone curves from

ASCE 41-13 (American Society of Civil Engineers, 2014).

For a given analysis, collapse is determined by three criteria: (1) global instability as indicated by a convergence

failure; (2) if any plastic deformations exceed the maximum values from the ASCE 41 backbone curves used in

the analysis; (3) if a non-simulated failure mechanism, such

as flexure-shear or shear-induced axial failure is determined to have occurred in any element (Applied

Technology Council, 2009; Aslani, 2005). The FEMA

P695 far-field earthquake record set, scaled to five different

IMs, is used for the analysis. The far field record set

consists of 44 records (22 earthquakes with two directions

each) from large magnitude earthquakes, selected to capture adequate record-to-record variability, for a total of

220 nonlinear time history analyses. For this study, the

spectral acceleration at the fundamental period of the

building, SaT1, is used as the intensity measure. Engineering demand parameters are denoted by θ, and are

typically maximum plastic hinge rotation for structural

components and maximum floor accelerations or interstory

drifts for nonstructural components. The maximum response outputs from these analyses are used to fit the prior

model of seismic demands.

The prior model of seismic demands estimates the

probability distribution for the maximum seismic demand on a component. This probability is conditioned on

an input IM and non-collapse, p(θi|SaT1, NC). For fitting

the model, only non-collapse NLTH analyses are considered, as the methodology proposed in this paper does

not apply for collapsed buildings. Therefore, the noncollapse case is implicit in the fit parameters. For simplicity in notation, the NC is dropped but is still implied

in subsequent formulations. Each demand is assumed to

be described by a lognormal distribution, parametrized

by the median demand, ^θ, and dispersion, β, (Applied

Technology Council, 2018a), which vary as functions of

SaT1. The lognormal distribution is estimated from responses from 220 NLTH analyses, based on the procedure developed by Miranda and Aslani (2003) and

applied in the Virtual Inspector methodology by MitraniReiser et al. (2016). The output model gives ^θ and β as

functions of SaT1 to estimate p(θi|SaT1) for an arbitrary

input intensity.

The lognormal distribution adequately describes many

types of EDPs but has limitations when the EDPs can take

on zero values. Maximum floor accelerations, interstory

drifts, and column axial loads, for example, will be strictly

positive for any earthquake intensity, and are well described by the lognormal distribution (Miranda and Aslani,

2003). Maximum plastic hinge rotation, however, will be

zero if the element does not yield. Such occurrences of zero

values cannot be adequately described with a lognormal

distribution. For plastic hinge rotation, therefore, a zeroinflated lognormal distribution is proposed that accounts

for the probability that the hinge will remain elastic.

A zero-inflated distribution describes a phenomenon

with two separate processes: the first describes the

Levine et al. 5

probability of an observation being zero and the second

describes the probability distribution of the nonzero values.

The zero-inflated lognormal distribution is applicable for

continuous, nonzero data where zero values are frequent

and meaningful (Belasco and Ghosh, 2008; Calama et al.,

2011). For example, zero-inflated models have been used to

predict tornado property damage, where two conditions are

important to consider: (1) the presence or absence of

damage and (2) the severity of the damage, if present (Diaz

and Joseph, 2019).

The binary zero/nonzero process is described with a

binomial distribution, which is fit to the analysis results

using a logistic regression (Calama et al., 2011). For each

hinge, the responses are considered for each ground motion

scaled to the same SaT1. The probability of a hinge having

zero plastic rotation given SaT1 is the percentage of those

ground motions for which the response is zero. The output

is a probability, γ, as a function of SaT1, that a particular

hinge will remain elastic or will behave nonlinearly. The

approach proposed by Miranda and Aslani (2003) is then

used to fit the lognormal component of the distribution to

the nonzero response values. The form of the distribution is

as follows

pðθijSaT1Þ ¼  γ if θi ¼ 0

ð1 γÞf ðθiÞ if θi > 0 (2)

where f (θi) is the lognormal PDF fit to the nonzero response values.

When the zero-inflated lognormal distribution is used,

the median ^θi is modeled using a sigmoid regression, of the

form

θi ¼ f ðIMÞ ¼ L

1 þ eðk×ðIMIM0ÞÞ (3)

where L, k, and IM0 are parameters to be optimized and

IM = ln(SaT1). Curve fitting is performed with SciPy

(Virtanen et al., 2020). The sigmoid curve is chosen because the regression model proposed by Miranda and

Aslani (2003) was consistently underpredicting hinge rotations at high earthquake intensities. With these parameters defined, the result is a complete prior distribution for

maximum plastic hinge rotations, column axial loads, floor

accelerations, and interstory drifts as a function of input

earthquake intensity.

Identification of damage-sensitive components. The prior

probabilities are used to select a subset of building components for post-earthquake assessment, termed damagesensitive components. External components are selected

based on their probability of experiencing damage; internal

components are selected based on their probability of

exceeding a Life Safety (LS) performance objective, based

on ASCE 41. In this context, the LS limit state is exceeded

when the component’s plastic hinge rotation exceeds the

ASCE 41 threshold (American Society of Civil Engineers,

2014). The rationale for this difference is that damage can

be directly observed on the exterior while building safety

can only be inferred from the unobserved internal components. Both probabilities are calculated at SaT1 corresponding to the maximum considered earthquake (MCE)

hazard level, per ASCE 7-16 (American Society of Civil

Engineers). Predicting damage requires a fragility model

for estimating damage state from the analysis model

demands.

The damage measures, DM, considered in this paper are

discrete, qualitative damage states (DS). Typical structural

components are assigned four possible damage states: no

damage (DS0), light damage (DS1), moderate damage

(DS2), and severe damage (DS3), while typical nonstructural components are assigned two possible damage

states: no damage (DS0) or damage (DS1). In the equations

that follow, DSij refers to the jth DS on component type i.

The probability of a component being in a particular

damage state is based on the difference between that

component’s probability of meeting or exceeding that

damage state and the probability of meeting or exceeding

the next highest damage state. For example, the probability

of DS1 is the fragility value of DS1 minus the fragility

value of DS2. The probability of meeting or exceeding

DS0 is always 1, and as the most severe state, the probability of being in DS3 is equal to the probability of

meeting or exceeding DS3.

Bayesian updating on exterior component

EDP models

The prior demand models, p(θi|SaT1), on exterior components are updated using a direct application of Bayes’

Theorem. Using Bayes’ Theorem, the posterior distribution, p(θ|SaT1, DSij), is as follows

p



θijSaT1, DSij

¼ P

DSij



θi, SaT1



pðθijSaT1Þ

P

DSij



SaT1

 (4)

Assuming that damage states conditioned on θi, P[DSij|θi

],

are independent of SaT1, the likelihood term, P[DSij|θi

, SaT1],

reduces to P[DSij|θi

] and can be computed from component

fragility curves for DSij. The total probability of observing

damage state DSij at the measured intensity, P[DSij|SaT1], is

given by equation (1), updated as follows

P½DSijjSaT1 ¼ Z

P½DSijjθi, SaT1pðθijSaT1Þdθi

¼

Z

P½DSijjθipðθijSaT1Þdθi

(5)

6 Advances in Structural Engineering 0(0)

The prior, p(θi|SaT1), is the model fit to the structural

response outputs in the previous section. The posterior

distribution is estimated directly by numerically computing

each component of Bayes’ Theorem. A lognormal distribution is fit to the posterior data to enable efficient sampling

in subsequent steps. No additional modifications are

needed for strictly positive EDPs like acceleration and drift.

Bayesian updating requires additional considerations

for the zero-inflated lognormal distribution used to model

plastic hinge response. For any DSij where j > 0, P[DSij|θi =

0, SaT1] = 0. In this case, γposterior = 0 because plastic

deformation must be nonzero if there is damage. When j >

0, only the lognormal portion of the model need be considered, and the formulation is unchanged. For j = 0, P

[DSi0|θi = 0, SaT1] = 1 and

γposterior ¼ γ

P½DSi0jSaT1 (6)

For the condition when θi > 0, the formulation is unchanged, but the computed posterior is scaled by (1

γposterior). At this point, the posterior distribution on exterior

components can be integrated to determine the probability

of exceeding a component-wise safety threshold. However,

the behavior of internal building components is not readily

visible from the exterior; this behavior is inferred from

exterior observations.

Surrogate modeling for interior component

EDP models

To predict demands on interior components, a surrogate for

the nonlinear analysis model is developed. A surrogate

model is a computationally efficient model that emulates

the response of a more complex model (Forrester et al.,

2008). Here, the surrogate stands in for the computationally

expensive finite element analysis. The surrogate model

allows rapid propagation of uncertainty; whereas the

nonlinear analysis model can take minutes to hours to

process a single realization, the surrogate can process

thousands of realizations in seconds. The surrogate models

used in this paper map the values of maximum plastic hinge

rotations on exterior components to maximum plastic hinge

rotations on interior components.

A different predictor is constructed for each interior

demand, mapping from a multidimensional space to a

scalar output. This approach has two advantages. First, it

allows for the use of a relatively simple predictor model;

second, it provides the flexibility to consider additional

outputs without having to retrain the entire model. Following the same motivations as for the zero-inflated

probability models, a two-step surrogate is considered.

The first step uses a support vector machine (SVM) that

classifies the output as zero or nonzero. An SVM is chosen

for the surrogate model because it is an efficient binary

classifier for high-dimensional data. If the SVM predicts a

nonzero output, the second step uses a neural network to

predict the maximum rotation on the interior hinge.

Each individual predictor uses the same SVM parameters and multi-layer artificial neural network architecture.

SVMs are supervised binary classifiers. A linear SVM finds

an optimal hyperplane that separates the two classes while

maximizing the minimum distance to the hyperplane for

either class. Nonlinear classification is enabled through the

use of nonlinear kernel functions (Boser et al., 1992). This

paper uses the scikit-learn SVM implementation in Python

with a radial basis function kernel (Pedregosa et al., 2011).

The network architecture used in this paper is defined in

Table 1. A ReLU activation function (Glorot et al., 2011) is

applied after all linear layers except the last layer. After

training a predictor for each output variable, the entire set of

predictors forms the surrogate model.

Correlated samples of maximum exterior hinge rotations are generated following the procedure described by

Burton and Deierlein (2018) based on Liu and Der

Kiureghian (1986) to generate a vector of correlated uniform random numbers. The correlation matrix is calculated

based on the response outputs from the NLTH analyses for

training the neural networks. After generating a vector of

correlated uniform random numbers, the inverse method is

applied to transform the correlated uniform random

numbers to set of correlated of random numbers distributed

according to the posterior distributions on exterior hinges.

In the final output vector, the entries are correlated according to the original correlation matrix, but each entry

follows the posterior distribution of the corresponding

hinge. Many realizations of maximum exterior plastic

hinge rotations are generated and propagated through the

surrogate model. The result is a set of samples from the

conditional distribution of maximum demands on each

interior component, conditioned on the visible damage to

exterior structural components.

The conditional distributions for maximum hinge rotation on both interior and exterior components are used to

classify the safety of the earthquake-affected building. For

example, the conditional distributions can be integrated to

estimate the probability of a component exceeding a LS

limit state, as defined in ASCE 41. If the probability exceeds a certain threshold, the building is unsafe. If the

probability is low, the building is likely safe to be reoccupied. The distributions can also be integrated with

fragility curves, as in the Virtual Inspector (Mitrani-Reiser

et al., 2016), to predict the distribution of damage

throughout the building. The following section demonstrates the application of this methodology on an example

building, starting with the results of an exterior UAV-based

inspection, and estimating the probability of exceeding a

LS limit state.

Levine et al. 7

Case study: Nonductile reinforced

concrete moment frame building

The methodology described in the previous section is

demonstrated on Turner Hall, which houses the Department of Crop Sciences at the University of Illinois in

Urbana, Illinois. A description of Turner Hall is provided,

and the modeling approach and assumptions are described.

A prior model of seismic component demands is developed, exterior and interior components are identified for

monitoring, and appropriate component fragility curves are

selected. Training procedures for the surrogate model are

presented, followed by the introduction of a validation

ground motion set for demonstrating the updating procedures. Finally, the results for probability updating on exterior components, demand inference on interior

components, and predictions for building safety are

compared against ground truth results from the validation

ground motions.

Building description

Turner Hall is a nonductile reinforced concrete moment

frame structure with a one-way flat slab floor system. It was

originally constructed in the 1960s and expanded in the

1970s. The building is five stories, with a penthouse and

basement. The moment frame is exposed on the eastern

elevation, making Turner an ideal candidate for UAV-based

inspections. (Figure 2(a)). There is a large rooftop air

handler unit (Figure 2(b)) in addition to a ground level tank

and mechanical unit (Figure 2(b)). Seismic design parameters, based on ASCE 7-16, are provided in Table 2. A

structural analysis model was created based on a review of

the original construction documents.

A nonlinear structural analysis model of Turner Hall is

developed based on the procedures described in Section

(Figure 2(c)). The model is created in SAP2000 (Computers and Structures, Inc.). Dead and live loads are estimated from a review of the original structural and

architectural design documents. The building is assumed to

behave as a reinforced concrete moment frame, with exterior nonstructural masonry walls detailed to provide no

contribution to lateral strength or stiffness. Nonlinear behavior is considered by assigning concentrated plastic

hinges at the ends of every beam and column. Key

modeling assumptions and parameters are as follows:

· Joint flexibility is accounted for implicitly, based on

ASCE 41 recommendations for the strong beamweak column condition present in the building. At

joints, rigid offsets are modeled at beams but not at

columns.

· Beam and column cracked stiffnesses are based on

NIST guidelines (NIST, 2017) and Kwon (2016). For

beams, the effective stiffness is taken as 25% of gross

stiffness; for columns the effective stiffness is taken

as 40% of gross stiffness.

· Beams are assumed to be flexure controlled, with

cyclic backbone curves selected for nonconforming

transverse reinforcing and low shear demands, per

ASCE 41-13.

· Columns are assumed to be flexure-shear controlled,

with cyclic backbone curves selected for high

transverse reinforcing and low shear demands, per

ASCE 41-13.

Using the nonlinear analysis model, an incremental

dynamic analysis (IDA) is performed to determine an

appropriate set of intensities for scaling the earthquake

records to fit a prior probability model. In an IDA, a ground

motion is scaled to increasing intensities and a NLTH

analysis is conducted at each intensity until collapse occurs

(Vamvatsikos and Cornell, 2002). Collapse is considered to

occur if the NLTH causes global instability or if any hinge

exceeds 0.06 radians, the maximum value in the backbone

curves for the hinges. The collapse intensity of an earthquake record is defined as the value of SaT1 of the scaled

record at which collapse occurs. All ground motions are run

parallel to the exposed moment frame in Figure 2. An IDA

is run for each record in the FEMA P695 far-field record set

to estimate a median collapse intensity for the building

SadT1 = 0.21 g. Based on this value, each record is scaled to

SaT1 = 0.05 g, 0.1 g, 0.15 g, 0.2 g, and 0.25 g to fit a prior

probability model of maximum seismic demands. Probability models are developed for plastic hinge rotations, floor

accelerations, interstory drift ratios, and column axial

loads.

To relate EDPs to visible damage, component fragility

curves are developed for all structural and nonstructural

components of interest on the building exterior based on

FEMA P58 guidelines (Applied Technology Council,

2018a). For structural components, a damage progression with three damage states is selected based on typical

progressions for nonductile RC moment frame columns in

FEMA P58 (Applied Technology Council, 2018c). These

damage states, along with the fragility curve parameters,

are listed in Table 3. Following Burton and Deierlein

(2018), the EDP for frame beams and columns is

Table 1. Neural network architecture.

Layer Input Output

Linear 1 32 16

Linear 2 16 16

Linear 3 16 16

Linear 4 16 1

8 Advances in Structural Engineering 0(0)

maximum plastic hinge rotation. The median values are

defined in proportions to the element’s capping rotation, θc,

the rotation corresponding to the maximum force in the

hinge (Applied Technology Council, 2009). Specific

fractions of θc for different damage states are taken from

Burton and Deierlein (2018). Capping rotations for each

beam and column are taken from the backbone curves used

in the structural analysis. For columns, θc is also a function

of the axial compression load, which is considered in

subsequent analysis.

Nonstructural damage is assigned binary damaged/

undamaged state. MEP components are considered

acceleration-sensitive, where damage is assigned based on

floor accelerations. Walls are acceleration-sensitive for outof-plane motion and drift-sensitive for in-plane motion.

Damage states and fragility curve parameters for nonstructural components are listed in Table 4. Information on

wall in-plane drift capacity, wall out-of-plane anchorage,

and MEP anchorages is lacking in the design documents.

Therefore, per FEMA P58 guidelines, fragility curves are

based on design requirements at the time of construction.

Due to the high uncertainty, a dispersion of β = 0.5 is

assigned for all nonstructural components. Median interstory drift ratio values for in-plane damage are estimated

based on FEMA P58-2 Section 2.5.1.2. For acceleration

sensitive damage states, the median EDP is estimated from

ASCE 7 code-based limit states for wall out-of-plane and

nonstructural equipment anchorage (American Society of

Civil Engineers) and FEMA P58-2 Section 7.3 guidelines

(Applied Technology Council, 2018b).

The prior probabilities and component fragility models

are used to identify a set of exterior beams and columns for

visual monitoring and a set of interior beams and columns

for evaluating post-earthquake safety. On the east façade of

Turner Hall, where the concrete moment frame is visible,

32 hinges are identified as having greater than 0.5% chance

of being damaged in an MCE-level earthquake, provided

the building does not collapse. The beams and columns

associated with these hinges are selected for visual monitoring. Each hinge, identified as half of a beam or column,

Figure 2. (a) East elevation of Turner Hall, showing the exposed moment frame. (b) Exterior MEP units. (c) Structural analysis model.

Table 2. Seismic design parameters for Turner Hall.

Parameter Value

Risk category III

Site class D (stiff soil)

S1 (mapped spectral acceleration at 1s period) 0.091 g

SM1 (MCE spectral acceleration at 1s period) 0.218 g

T1 (fundamental period of building) 2.24 s

SaT1 at MCE 0.1 g

Median collapse intensity, SadT1 0.21 g

Table 3. Structural damage states and associated fragility curve parameters.

Damage

state Description EDP Median EDP Dispersion

DS1 Light Residual cracks > 0.06 in; no significant spalling or exposed

reinforcing

Maximum plastic

hinge rotation

0.3θc (beams)

0.25θc

(columns)

0.4

DS2

Moderate

Spalling of concrete cover; possible exposure of transverse

reinforcing but not longitudinal reinforcing

Maximum plastic

hinge rotation

0.7θc (beams)

0.55θc

(columns)

0.4

DS3 Severe Spalling exposes longitudinal reinforcing; possible concrete core

crushing; possible fracture or buckling of reinforcing

Maximum plastic

hinge rotation

1.0θc (beams)

0.8θc

(columns)

0.4

Levine et al. 9

is shown in Figure 3. On the interior of the building,

30 hinges are selected for evaluating post-earthquake

safety, based on a greater than 1% chance of exceeding

a LS limit state in an MCE-level earthquake. These hinges

are shown in Figure 4. For both exterior and interior, the

distribution of hinges is asymmetric. This asymmetry is a

consequence of the asymmetric and irregular building

geometry (several frame columns are discontinuous) and

torsional behavior of the building. Next, an SVM and

neural network are trained to map from the 32 exterior

hinges to each of the 30 interior hinges to create the surrogate model.

Surrogate model training

A combination of the FEMA P695 far-field record set and

synthetic ground motions is used as training data for the

surrogate models. For each training ground motion, the

maximum plastic rotation for hinge is recorded. The rotations for the 32 exterior hinges are used as predictor

inputs, and the rotations for the 30 interior hinges are the

outputs. Each record in the far-field set is scaled to 0.5, 1.0,

1.5, 2.0, 2.5, 3.0, 3.5, and 4.0 times the base record. Each

record is additionally scaled by the record-specific scaling

factor for normalizing peak ground velocity, per FEMA

P695. Additionally, 600 synthetic ground motions are

randomly generated using the Kanai-Tajimi spectrum

(Bogdanoff et al., 1961; Kubo and Penzien, 1976). Of these

training ground motions, 467 did not cause collapse and are

used to train the surrogate model. Of these 467, 374 ground

motions are used as a training set and the remainder is used

for validation during training.

Training each predictor is a two-step process; first, the

SVM is trained to predict zero or nonzero outputs, then the

neural network is trained on nonzero outputs only. Average

accuracy over all 30 SVM predictors is 96.7% for both

training and validation. Each neural network is created and

trained with PyTorch (Paszke et al., 2019) using a smooth

L1 loss function (Torch Contributors). Initially, each network was trained for 10,000 epochs. Depending on the

performance, select networks were retrained for 20,000 or

30,000 epochs. Examples of true versus predicted values

are plotted for training and validation sets in Figure 5.

Ideally, the points should follow the 1:1 slope on the dashed

line, indicating that the predictions match true values.

Validation dataset and damage simulation

A new set of ground motions is selected to test the full

methodology described in this paper. Six earthquake records from the PEER NGA-West2 database, each with two

horizontal ground motions, are selected for this validation

(Ancheta et al., 2013). The records are scaled to 1.0, 1.5,

2.0, 2.5, and 3.0 times the base record, for a total of

60 NLTH analyses. Of these 60 analyses, 16 caused collapse, for a total of 44 non-collapse validation analyses.

The maximum hinge responses from each NLTH are used

to query component fragility curves and generate randomized realizations of damage. A damage realization is a

list of fragility-consistent damage states for each of the

32 exterior hinges. 100 damage realizations are generated

for each of the 44 scaled non-collapse ground motions, for a

total of 4400.

For each damage realization, the prior demand distributions for exterior hinges are updated based on the assigned damage state. The posterior distributions are

estimated with numerical integration, so a lognormal

distribution is fit to each of the resulting posterior distributions. The median and dispersion of the lognormal

distribution are estimated by numerically integrating the

posterior. The correlation matrix for sampling exterior

hinge values is calculated from the surrogate model

training data. For each damage realization, 10,000 realizations of maximum hinge rotation are generated and input

to the surrogate model. The outputs of the surrogate

characterize the distributions of maximum rotations on

internal hinges and are used to subsequently predict the

safety of the building.

Safety assessments are based on performance objectives

defined by ASCE 41. For the purposes of this study, safety

is considered in terms of the LS performance objective. To

meet LS performance, there may be moderate damage to

structural components, but the gravity load carrying system is intact and there is some residual lateral strength

and stiffness. Overall, the risk of life-threatening injury due to structural or nonstructural damage is low

Table 4. Nonstructural damage states and associated fragility curve parameters.

Component Damage description EDP Median EDP Dispersion

MEP Sliding or overturning Maximum floor

acceleration

Varies by site

location

0.5

Walls (in-plane) In-plane shear cracking; possible partial collapse Maximum interstory drift

ratio

0.017 0.5

Walls (out-ofplane)

Out-of-plane two-way bending cracks; possible partial

or total collapse

Maximum floor

acceleration

Varies by site

location

0.5

10 Advances in Structural Engineering 0(0)

Figure 3. East exterior elevation of Turner Hall. Components identified for visual monitoring are highlighted in red.

Figure 4. East exterior elevation of Turner Hall. Components identified for visual monitoring are highlighted in red.

Figure 5. Training and validation predictions for a typical hinge compared to the ground truth values. The dashed line has a 1:1 slope.

Levine et al. 11

(American Society of Civil Engineers, 2014). Life safety

does not perfectly correspond to a green tag; in fact, the

SPUR shelter-in-place performance objective, which indicates a building is safe enough to be occupied even if all

services have not been restored, is more stringent than the

ASCE 41 LS limit state (SPUR, 2012). Nevertheless, the

LS objective is a representative objective suitable to

demonstrate the proposed methodology. Performance is

assessed based on the plastic rotation in each nonlinear

hinge, using acceptance criteria from ASCE 41-13 Table

10-7 for beams and Table 10-8 for columns. For the validation ground truth earthquakes, each hinge’s performance

is determined by whether the plastic rotation exceeds the

LS acceptance threshold. The building is labeled unsafe if

any hinge exceeds LS. A probability of exceeding LS is

reported based on the output distributions. For columns, the

LS acceptance threshold is a function of the column

compression load. Column loads can be taken directly from

the analysis model for the ground truth. For the probability

models in this study, the column compression load is

considered deterministic and is taken as the median

maximum compression load at SaT1 for the earthquake.

Results and discussion

This section presents the results of the application of the

framework described in the preceding sections. First, results for exterior components are presented. For structural

components, both the prior distributions conditioned on

SaT1 alone and the posterior distributions conditioned on

observed damage are compared against true responses from

the structural analysis. Results of the probability updating

procedure are also presented for drift- and accelerationsensitive nonstructural components. Second, the prior

distributions for interior plastic hinge rotations conditioned

on SaT1 alone and the conditional distributions inferred

using the surrogate model are compared against true responses from the structural analysis. This comparison

demonstrates that the proposed method improves predictions made with the prior distribution alone.

Evaluation metrics

Several metrics of success are considered for both exterior

and interior hinges. For each hinge for each of the

100 damage realizations, the results from all 44 validation

ground motions are considered. To assess the overall

performance, the slope of the best fit line and the correlation coefficient between the median of the distribution

(prior or posterior/conditional) and the true value is calculated. The results are considered to improve over the

prior distribution if the slope of the best fit line is closer to

1, indicating better correspondence with true values.

Similarly, if correlation between the median of the output

distributions and the true values is higher than the correlation between the median of the prior and the true values,

this is considered an improvement.

Posterior distributions on exterior

structural components

Figure 6 shows several examples of results for representative exterior hinges. Each example shows the prior distribution, p(θ|SaT1), the posterior distribution, p(θi|DSij,

SaT1), and the ground truth value from the structural

analysis. Each example has two subplots. The first showsγi,

the probability mass at θi = 0. The second shows the PDFs

for θi ≠ 0. Note that these PDFs integrate to (1 γi), rather

than 1.

Given the uncertainties, performance varies based on

hinge, earthquake, and individual damage realization. To

consider overall performance, for each realization on each

hinge, the predictions of all 44 earthquakes are considered.

Figure 7 shows the median of the prior and posterior

distributions of maximum hinge rotation plotted against the

ground truth value. For both sets of points, the best fit slope

and correlation coefficient are reported. A best fit slope

closer to 1 indicates better performance. For each hinge,

prediction quality improvement is defined as a slope for the

posterior medians closer to 1 than for the prior medians.

Prediction quality is defined to be similar if |mposterior

1| |mprior 1| ≤ 0.01, where m is the best fit slope reported in Figure 7. Prediction quality is considered worse if

|mposterior 1| |mprior 1| > 0.01. For each hinge, the

fractions of improved, similar, and worse realizations are

reported in Figure 8. For 28 out of 32 exterior hinges,

performance of the posterior distribution is the same or

improved over the prior for more than half of all

realizations.

The objective of probability updating on exterior hinges

is to incorporate damage observations from an exterior

UAV survey of an earthquake-affected building into a

performance-based safety assessment. For exterior building components, estimates of maximum earthquake demands on exterior components are updated based on

observed damage states through direct application of

Bayes’ Theorem. Figure 8 shows that in general, the

posterior distributions better predict the ground truth value

relative to the prior.

Figure 6 illustrates the effectiveness of the updating

procedure. In Figure 6(a), the prior underpredicts the

ground truth, and the posterior corrects for the underprediction. The posterior P[θi = 0|DSij] correspondingly

drops to zero due to the observation of damage. Similarly,

in Figure 6(b), the posterior compensates for the prior’s

overprediction of the ground truth. The posterior P[θi = 0|

DSij] correspondingly increases. Figure 6(c) is an example

12 Advances in Structural Engineering 0(0)

where the probability of observing the documented damage

state is so high that there is little change between prior and

posterior. However, in cases where the observed damage is

unlikely, the median may move away from the true value,

as illustrated in Figure 6(d). While Figure 6(d) does show

that performance decreases for the posterior in certain

cases, Figure 8 shows that overall, performance typically

improves for the posterior over the prior distributions.

Figure 6 also illustrates the utility of modeling hinge

rotations with the zero-inflated lognormal distribution. The

binary zero/nonzero probability is a good indicator of the

success of the updated distributions relative to the priors, as

indicated in Figure 6(a) and (b), where the relative changes

from prior P[θi = 0] to P[θi = 0|DSij] correspond well with

the observation of damage. For sufficiently high earthquake intensities (Figure 6(c) and (d)), the prior and

posterior are zero or near zero, serving as an immediate

indicator that the building has behaved nonlinearly and that

damage may be present throughout the structure.

Posterior distributions on exterior

nonstructural components

Probability models for maximum floor acceleration and

interstory drift are updated based on observed nonstructural

component damage. Results of the updating procedure are

reported for 3 earthquakes at low, moderate, and high

intensity, corresponding to SaT1 = 0.0484 g, 0.159 g, and

0.245 g. Figure 9 shows hypothetical results for updating

models for maximum floor acceleration given an undamaged component (DS0) or given a component damaged by

overturning or sliding (DS1). Results are reported for MEP

components attached at the ground level and at the roof

level. Wall damage corresponds with either interstory drift,

for in-plane motion, or floor acceleration, for out-of-plane

motion. Figure 10 shows results for maximum drift demands at representative stories.

The results for nonstructural components (Figures 9 and

10) show the effectiveness of the probability updating

procedures for nonstructural components. The results

generally conform with expectations: when damage is

observed, the posterior distribution predicts higher values;

when no damage is observed, the posterior distribution

predicts lower values. The utility of updating models for

interstory drift and floor accelerations based on nonstructural damage is that these are global metrics rather than

local metrics, like plastic hinge rotation. The results can

therefore give a general sense of the scale of damage at a

particular story. Furthermore, many buildings do not have

exposed structure on the exterior, so the ability to infer

demands based on nonstructural observations is important.

However, as illustrated in Figures 9 and 10, both the prior

and posterior distributions are poor predictors of the ground

Figure 6. Example results for probability updating on exterior hinges. For each example, the bar chart show the probability mass at θi =

0. The remaining probability density function for nonzero values of θi is shown. The dashed lines are the ground truth values from the

structural analysis.

Levine et al. 13

truth value from the structural analysis, and the posterior

predictions are not consistent relative to the ground truth

values. Additionally, in some cases, no information is

gained from the updating procedure because the prior is

nearly identical to either posterior (e.g., stories 1 and 3 in

Figure 10). Based on these results, maximum demands

estimated from nonstructural component observations are

excluded from the surrogate model. This performance is

Figure 7. Example results for probability updating on exterior components. In each plot, the median of the prior and posterior

distributions are plotted against the ground truth values for a given hinge for all 44 earthquakes. The best fit slope and correlation

coefficient (R) are shown for each set of points. In each plot, the line y = x, with an idealized 1:1 slope, is shown.

Figure 8. Percentage of improved, similar, or worse performance plotted for each exterior hinge.

14 Advances in Structural Engineering 0(0)

likely because the fragility curves are based on assumptions

about building code design requirements rather than an

analysis of attachment details. Uncertainty in the attachment

design necessitates high values of dispersions in the fragility

models. If nonstructural component demands are to be used

as part of the updating and inference procedure, therefore,

the fragility curves must be developed based on a review of

construction details rather than design demand assumptions.

Figure 9. Probability of maximum floor accelerations for different input earthquake intensities, based on observed MEP damage at the

ground level and roof level. For each case, the posterior distribution is plotted conditioned on no damage observed (DS0) or damage is

observed (DS1). The dashed lines are the ground truth values from the structural analysis.

Figure 10. Probability of maximum interstory drifts for different input earthquake intensities, based on observed wall damage at

different stories. For each case, the posterior distribution is plotted conditioned on no damage observed or damage is observed (DS1).

The dashed lines are the ground truth values from the structural analysis.

Levine et al. 15

Conditional distributions on interior

structural components

Figure 11 shows several examples of results for representative interior hinges. Each example shows the prior

distribution, p(θi|SaT1), the conditional distribution inferred

from the surrogate model, p(θi|Exterior Observations,

SaT1), and the ground truth from the structural analysis.

Each example has two subplots. The first shows γi, the

probability mass at θi = 0. The second shows the PDF for θi

≠ 0. Note that these PDFs integrate to (1 γi), rather than 1.

To consider aggregate performance, for each realization

on each hinge, the predictions of all 44 earthquakes are

considered. Figure 12 shows the median of the prior and

inferred conditional distributions of maximum hinge rotation plotted against the ground truth value. Definitions of

improved, similar, and worse performance are as for exterior hinges. For each hinge, the fractions of improved,

similar, and worse realizations are reported in Figure 13.

For 25 out of 30 interior hinges, performance of the

posterior distribution is similar or improved over the prior

for more than half of all realizations.

An exterior building survey may be insufficient to

characterize the safety of an earthquake-affected building;

the objective is therefore to predict performance of interior, nonvisible components based on exterior observations. To this end, a surrogate model is developed that

takes demands on exterior components as input to predict

demands on interior components. Demand realizations are

sampled from the posterior distributions on exterior

components and propagated through the surrogate model.

The result is a distribution that characterizes maximum

demands on interior components, conditioned on the

exterior observations. The success of the method is in part

due to correlated sampling, which results in a more

physically realistic realization of demands on exterior

components. Figure 13 shows that in general, the conditional distributions better predict the ground truth value

relative to the prior.

The surrogate model performs well at predicting

maximum demands on interior frame components. In

particular, the high accuracy of the SVM suggests that it is a

reliable method of estimating whether post-elastic behavior

has initiated on a component. Figure 5 shows that the

neural network predictions closely match the ground truth

values. The combined SVM and neural network surrogate

shows promise for rapidly predicting demands on components throughout a building.

Figure 11 illustrates the effectiveness of the full updating procedure. Figure 11(a)–(c) show examples of the

output distribution correcting for inaccurate prior distributions. As with the exterior components, the utility of

modeling hinge rotations with the zero-inflated lognormal

distribution is apparent. For lower intensity earthquakes

(Figure 11(a)), the increase in P[θi = 0|Exterior Damage,

SaT1] relative to P[θi = 0|SaT1] immediately suggests that

demands on that component are lower than would have

been expected based on intensity alone. Overall, for both

Figure 11. Example results for probability updating on interior hinges. For each example, the bar chart show the probability mass at θi =

0. The remaining probability density function for nonzero values of θi is shown. The dashed lines are the ground truth values from the

structural analysis.

16 Advances in Structural Engineering 0(0)

exterior and interior components, the comparisons presented confirm that predictions of maximum hinge rotations are indeed improved with the updated distributions.

Furthermore, these predictions demonstrate that postearthquake damage observations can be successfully

integrated into a structural assessment of an earthquakeaffected building.

Nevertheless, certain components do not show improvement with the proposed updating procedure. For

example, at hinges 593H1, 605H1, and 629H1, the updated

Figure 12. Example results for probability updating on interior components. In each plot, the medians of the prior and inferred

distributions are plotted against the ground truth values for a given hinge for all 44 earthquakes. The best fit slope and correlation

coefficient (R) are shown for each set of points. In each plot, the line y = x, with an idealized 1:1 slope, is shown.

Figure 13. Percentage of improved, similar, or worse performance plotted for each interior hinge.

Levine et al. 17

distribution consistently performs worse than the prior.

This performance, at least in part, can be explained by the

quality of surrogate model predictions. Training results

for each of the hinges is shown in Figure 14. While the

results generally appear good, note that for higher values

of true hinge rotation for 593H1 and 605H1, the neural

networks tend to underpredict the true values. For other

cases, such as 629H1, the surrogate training results

generally appear good. Here, poor performance may be a

result of high variability in the hinge’s response in the

analysis model, compounded by additional uncertainties

in the exterior hinge demands, fragility models, and surrogate models.

Combined with the posterior estimates on exterior

components, the updated demands on interior components

provide a complete picture of the demands throughout the

building. These updated distributions for component demands can be used in the Virtual Inspector (Mitrani-Reiser

et al., 2016), which uses a model of component demands to

estimate the probability of damage. Thus, the tagging

probabilities output by the Virtual Inspector can be refined

to account for the observed exterior damage. This study

enables the equivalent of a detailed ATC-20 evaluation

through the Virtual Inspector, which considers the interior

of the building in addition to the exterior. Furthermore,

several studies have proposed using machine learningbased classifiers (Burton et al., 2017; Zhang and Burton

2019; Zhang et al., 2018) to predict a building’s residual

post-earthquake collapse capacity based on structural responses, such as plastic hinge rotation. The results, when

integrated with a UAV-based automated data collection

program and the classifiers proposed by Burton et al.

(2017), Zhang et al. (2018), and Zhang and Burton

(2019), could produce a fully automated collapse analysis of an earthquake damaged structure. The next section

considers how the updated distributions produced by this

study are used to estimate the safety state of the damaged

building.

Building safety predictions

The posterior distributions on exterior hinges and inferred

conditional distributions on interior hinges are used to

estimate the probability of exceeding the LS performance

objective. These values are compared against ground truth

values from the validation ground motions. Of the 44 validation ground motions, 12 exceed LS performance and are

classified as unsafe based on the ground truth analyses.

Additionally, the observed damage states come from an

exterior UAV survey. Therefore, to show the utility of inferring interior demands, the classification considers whether

the LS threshold is exceeded by exterior visible components

or only by interior components. Of the 12 earthquakes resulting in unsafe performance, 9 are classified as unsafe based

on demands on exterior components, while 3 are classified as

unsafe based on interior components. For those 9 cases

classified as unsafe based on exterior components, LS is

exceeded by both exterior and interior components.

To illustrate the quality of predictions, results are shown

for 3 different ground truth cases. Figures 15 and 16 show

the results for each hinge for each case. In the first case,

shown in Figure 15, the building does not exceed the LS

performance objective. The first case is a high intensity

earthquake, with SaT1 = 0.15 g (Figure 15). In the second

and third cases, shown in Figure 16, the building exceeds

the LS performance objective and is therefore unsafe. In the

second case, the building exceeds LS based on both exterior and interior component performance (Figure 16(a)).

This case is the highest intensity of the 4 cases, with SaT1 =

0.18 g. In the third case, the building exceeds LS only when

considering interior component performance (Figure 16(b)).

This case has a lower intensity than the second case, with

SaT1 = 0.16 g. For each case, Figures 15 and 16 show the

distribution of predictions of the probability of exceeding LS

for all 100 damage realizations for each hinge. Each subplot

corresponds to a single earthquake. Plots in red indicate that

the ground truth value exceeds the LS threshold.

Figure 14. Examples of training results for hinges that did not show improvement. In each plot, the line y = x, with an idealized 1:1 slope,

is shown.

18 Advances in Structural Engineering 0(0)

In general, hinges indicated as unsafe in the ground truth

correspond with higher estimated probabilities of exceeding LS. For an individual hinge, the probability of

exceeding LS correlates with the intensity of the earthquake. This trend is evident with the two unsafe examples,

where Figure 16(a), which is a higher intensity earthquake

than Figure 16(b), has higher predicted probabilities of

exceeding LS.

Interior hinges typically show higher unsafe probabilities. This trend explains why for moderate level earthquakes (Figure 16(b)), exterior observations alone may be

insufficient to classify the building. While this trend may

change based on the characteristics of an individual

building, it reinforces the importance of the objective of

this study to predict interior component behavior based on

an exterior survey.

The final safe/unsafe classification is complicated by the

variability in response to individual earthquakes. For example, hinge 486H2 has a safe ground truth value in

Figure 16(a). However, the same hinge has an unsafe

ground truth value in Figure 16(b), despite having lower

predicted probabilities of exceeding LS. Similarly, hinge

665H1 has an unsafe ground truth value in Figure 16(a), but

overall has lower predicted unsafe probabilities than

486H2, which has a safe ground truth value for that

earthquake. Nevertheless, the objective is to assign a safety

state for the entire building, rather than an individual hinge.

Considering the global building safety, the building is

classified as unsafe if any hinge exceeds LS. Defining an

acceptable threshold for safety, such as a 25% probability

of exceeding LS on any hinge, will likely produce both

false negatives and false positives. For unsafe cases,

however, the predicted probabilities of exceeding LS for

any hinge are typically higher, regardless of the ground

truth safety. Therefore, even if an individual hinge is

misclassified, the building is likely to be classified

correctly.

This trend is illustrated in Figure 17, where probabilities

of exceeding LS are plotted against each other for several

random hinges. Each point is color coded by the global

ground truth safety state. Here, an unsafe label is assigned if

any building component in that time history analysis exceeds LS. Safe and unsafe points tend to cluster towards the

low and high probability regions, respectively. This observation suggests the possibility of training a classifier to

predict if the building has exceeded the LS limit state,

based on each individual component probability of exceeding LS. To assess this possibility, an SVM is trained

that takes the 62 component probabilities of exceeding LS

(32 exterior + 30 interior hinges) and outputs a safe/unsafe

label. The training ground truth is labeled as safe if no

hinges in exceed LS, and unsafe otherwise. SVM performance is evaluated with 10-fold cross-validation. The set of

44 validation ground motions are randomly partitioned

60% for training and 40% for testing. All 100 damage

realizations are considered for each ground motion. The

average accuracy for all 10 random dataset partitions is

95.1%. More critically, average training recall, defined as

true positives/(true positives + false negatives), is 89.6%,

indicating that there are very few false negatives where an

unsafe building would be misclassified as safe. These results shows promise for interpreting global building safety

from component results.

Nevertheless, the classification of global performance

based on individual components has been identified as a

weakness in the ASCE 41 approach to seismic assessment.

Specifically, the link between component demands and global

building performance is ambiguous (Deierlein 2011). This

Figure 15. Probability of exceeding life safety for each hinge for a high intensity safe case. Each plot represents an individual earthquake.

For each hinge, the distribution of predictions of the probability of exceeding life safety is plotted for all 100 damage realizations.

Levine et al. 19

Figure 16. Probability of exceeding life safety for each hinge for unsafe cases. Red indicates the ground truth hinge rotation exceeds life

safety. (a) High intensity earthquake where both interior and exterior hinges exceed life safety. (b) Moderate intensity earthquake

where only interior hinges exceed life safety. Triangles are the probability of exceeding life safety based on the predictions from an

unmanned aerial vehicle survey.

Figure 17. Probabilities of exceeding life safety for various hinges. In each plot, results from all 4400 damage realizations are plotted.

Points are color coded based on the global structure ground truth safety.

20 Advances in Structural Engineering 0(0)

weakness suggests that using component demands to predict

collapse capacity (Burton and Deierlein, 2018; Burton et al.,

2017; Zhang and Burton, 2019; Zhang et al., 2018) would

give a better sense of building performance than comparing

each component to a safety threshold. However, as noted

previously, the results are strongly affected by compounding

uncertainties in component-level demands, fragility models,

and global structural response. Using the updated distributions of component demands to estimate reductions in collapse capacity would introduce an extra layer of uncertainty in

the results. Additionally, even though decisions are made at

the component level, the surrogate model approximates the

physics of the building, and therefore incorporates the global

structural response. Furthermore, the goal of the proposed

methodology is to enable rapid classification of buildings in a

post-earthquake environment to assess immediate safety

threats. If any one component has exceeded LS, that component’s loss in capacity indicates a high risk of localized

component damage and heavy nonstructural damage that

would be a danger to anyone who entered the building. After

quickly establishing initial safety sufficient for shelter-inplace, subsequent analyses can consider global collapse

performance to evaluate repair, retrofit, and replacement

schemes for the damaged building.

Integration with a computer vision-based inspection

scheme

A key component of the rapid assessment framework

presented in this paper is the automated collection and

processing of inspection data using UAVs and computer

vision. This section illustrates how a UAV-based data

collection method can be integrated with the performancebased assessment methodology in the previous sections.

Levine and Spencer (2022) proposed a method to use a

BIM to assign fragility-consistent damage states to

building components based on a photographic UAV survey

of a building. Figure 18 summarizes the results of this

method applied to Turner Hall. Because Turner Hall is

intact and located in a low-seismicity region, a computer

graphics model is created of a hypothetical earthquakedamaged Turner Hall. Damage is simulated based on the

response from a NLTH analysis using the same earthquake

record as shown in Figure 16(b). Full discussion of these

results and the methods for generating the graphics model

are found in Levine et al. (in review). Figure 18(a) shows

several example images from a simulated UAV survey in

the Turner Hall graphics model. In Figure 18(b), one of the

UAV images (top image) is input into a semantic segmentation algorithm to detect visible structural damage

(second image). The BIM is overlayed on the image to

create a mask for the component of interest (third image),

so that only damage on that component is considered

(fourth image). Finally, the damage state is classified based

on the identified damage: light (DS1) if cracking is present,

moderate (DS2) if spalling or transverse rebar is present,

and severe (DS3) if longitudinal rebar is present. The true

and predicted damage states are shown in Figure 18(c).

While the predictions are generally accurate, several

damage states are overpredicted. For one hinge at the

penthouse level, the true moderate damage state is underpredicted as light damage.

The predicted states are input to the performance-based

assessment framework developed in the previous sections.

The models for prior demands, conditioned on SaT1 only,

Figure 18. Visual inspection results from a simulated unmanned aerial vehicle survey of Turner Hall. (a) Synthetic unmanned aerial

vehicle survey images of Turner Hall. (b) Procedure for assigning damage states to an example column, using a semantic segmentation

algorithm for damage identification and a building information model for component identification. (c) True damage states and damage

states predicted from unmanned aerial vehicle images.

Levine et al. 21

are updated based on the observed damage states for exterior hinges. Correlated samples are drawn from these

posterior distributions and propagated through the surrogate model to characterize the conditional distributions on

interior hinges. The probability of exceeding LS is estimated for each hinge. These probabilities are plotted as

triangles in Figure 16(b). Finally, the individual hinge

probabilities are input to the SVM predictor proposed

previously to assign an overall damage state. The final

prediction is unsafe, which agrees with the ground truth

unsafe value for this earthquake. Thus, the methodology

proposed in this paper enables automated decision making

about the safety of an earthquake-affected building based

on an exterior photographic UAV survey.

The procedures demonstrated in this paper can also

integrate with existing methods for automated inspection

and performance-based building assessments. For example, the goal of the BIM-based digital twin proposed by

Levine and Spencer (2022) is to link damage automatically

identified in UAV survey images to specific building

components. Damage information can be automatically

collected for updating the probability models in this study.

The procedures for updating and inferring component EDP

probability models based on observations of exterior

damage can directly integrate with previously developed

frameworks for performance-based post-earthquake tagging methods, such as the Virtual Inspector (Mitrani-Reiser

et al., 2016) and its extensions (Burton and Deierlein,

2018). Both the Virtual Inspector and this study use

similar procedures for estimating prior component demand

probability distributions. This results of this study demonstrate that these initial estimates can be refined using

post-earthquake damage observations obtained from UAVs

and computer vision methods.

Conclusions

This paper proposed and demonstrated a method for assessing building performance from an exterior UAV survey. There are two main steps to meeting this objective.

First, prior probabilities of maximum seismic demands on

exterior components were updated based on observed

damage states and component fragility models. This procedure was also applied to nonstructural components to

estimate building-scale demands, such as interstory drift

and floor accelerations. Second, a computationally efficient

surrogate model was developed to predict maximum demands on interior structural components. The surrogate

enables rapid propagation of uncertainty to characterize

distributions of maximum seismic demands on interior

structural components, conditioned on external damage

observations. These procedures were demonstrated on an

example building, Turner Hall, using a set of validation

earthquakes. The demonstration showed that the

probability models conditioned on exterior observed

damage more accurately predicted ground truth values

from a structural analysis. Additionally, the results provide

a way to improve existing PBEE-based post-earthquake

assessment methods by incorporating information from

observed damage to update models of component demands. For each validation earthquake, the estimated

distributions on both interior and exterior structural components of Turner Hall were used to estimate the probability of exceeding a target safety performance objective.

The results indicate the importance of the proposed

methodology for characterizing the demands on interior

components: in certain cases, the building can only be

classified unsafe based on the performance of interior

structural components.

Several contributions enable the proposed methodology.

A zero-inflated lognormal distribution was used to characterize maximum plastic hinge rotation, which more

adequately models demands that can take on zero values

compared to a lognormal distribution. The parameter for

the probability of zero plastic hinge rotation, γi, was found

to be a good indicator of damage within a building. A

machine learning-based model was developed as surrogate

for the structural analysis model. The high accuracy

achieved by the SVM and neural network surrogate shows

promise for representing complex nonlinear structural

analysis models for use in structural health monitoring

applications. Furthermore, despite high variability in the

LS probability predictions for individual hinges, it was

found that using a binary classifier, such as an SVM,

showed promise for aggregating individual hinge estimates

to classify the safety state of the building.

Nevertheless, the methods developed in this paper offer

several opportunities for improvement. In this study,

nonstructural damage information was not incorporated

into safety classifications because of the high uncertainty in

the fragility models. While this approach works for Turner

Hall, which has exposed structure, many buildings lack

exterior visible structural components. Therefore, future

research efforts will investigate the potential benefit of

improving nonstructural component fragility models by

conducting a thorough analysis of components’ attachments to the structure. At the structural scale, the methods

in this paper assume that the structural analysis model,

which was based on design documents, is representative of

the true structural behavior. Thus, no model uncertainty

was considered. The concentrated plastic hinge model used

in this paper is intended to approximate true structural

behavior; therefore, the hinge parameters will likely need to

be calibrated. Future research efforts will investigate using

observed exterior damage to update and calibrate the

structural analysis model. The calibrated model could then

be used to predict demands and damage throughout the

building, both on the interior and exterior.

22 Advances in Structural Engineering 0(0)

The methods established in this paper, when combined

with automated data collection and computer vision tools,

form a basis for a fully automated post-earthquake building

assessment system. When combined with an automated

data collection, damage detection, and damage classification framework, the proposed methodology describes a

complete system for rapidly predicting post-earthquake

building safety. Such a system will assist with rapid

community recovery after a major earthquake.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support

for the research, authorship, and/or publication of this article:

Financial support for this research was provided in part by the US

Army Corps of Engineers through a subaward from the University

of California, San Diego.

ORCID iDs

Nathaniel M Levine  https://orcid.org/0000-0003-0351-1546

Yasutaka Narazaki  https://orcid.org/0000-0002-1680-5079

References

American Society of Civil Engineers (2017) Minimum Design

Loads and Associated Criteria for Buildings and Other

Structures ASCE/SEI 7-16. ASCE Standard. Reston, VA:

American Society of Civil Engineers.

American Society of Civil Engineers (2014) Seismic Evaluation

and Retrofit of Existing Buildings : ASCE/SEI 41-13. ASCE

Standard. Reston, VA: American Society of Civil Engineers.

Ancheta TD, Darragh RB, Stewart JP, et al. (2013) Peer NgaWest2 Database. Technical report. Berkeley, CA, USA:

Pacific Earthquake Engineering Research Center.

Applied Technology Council (1989) ATC 20 Procedures for

Postearthquake Safety Evaluation of Buildings. Redwood

City, CA: Applied Technology Council (ATC).

Applied Technology Council (1995) ATC 20-2 Addendum to the

ATC-20 Postearthquake Building Safety Evaluation Procedures. Applied Technology Council (ATC).

Applied Technology Council (2009) FEMA P695 Quantification

of Seismic Performance Factors. 1st ed. Redwood City, CA:

Federal Emergency Management Agency.

Applied Technology Council (2018a) FEMA P-58-1 Seismic

Performance Assessment of Buildings Volume 1 – Methodology. September, 2nd ed. Redwood City, CA: Federal

Emergency Management Agency. URL https://www.fema.

gov/media-library-data/1396495019848-

0c9252aac91dd1854dc378feb9e69216/FEMAP-58_

Volume1_508.pdf

Applied Technology Council (2018b) FEMA P-58-2 Seismic

Performance Assessment of Buildings Volume 2 – Implementation Guide. 2nd ed. Redwood City, CA: Federal

Emergency Management Agency.

Applied Technology Council (2018c) FEMA P-58-3 Seismic Performance Assessment of Buildings Volume 3 – Supporting

Electronic Materials and Background Documentation. Redwood City, CA: Federal Emergency Management Agency.

Aslani H (2005) Probabilistic Earthquake Loss Estimation and

Loss Disaggregation in Buildings. PhD Thesis, Stanford

University, Stanford, CA USA.

Belasco EJ and Ghosh SK (2008) Modeling censored data using

mixture regression models with an application to cattle

production yields. 2008 Annual Meeting, July 27-29, 2008,

Orlando, Florida 6341, American Agricultural Economics

Association. DOI: 10.22004/ag.econ.6341. URL https://

ideas.repec.org/p/ags/aaea08/6341.html

Bogdanoff JL, Goldberg JE and Bernard MC (1961) Response of

a simple structure to a random earthquake-type disturbance.

Bulletin of the Seismological Society of America 51(2):

293–310. DOI: 10.1785/BSSA0510020293

Boser BE, Guyon IM and Vapnik VN (1992) A training algorithm

for optimal margin classifiers. In: Proceedings of the Fifth

Annual Workshop on Computational Learning Theory,

COLT ’92. New York, NY, USA: Association for Computing

Machinery, 144–152. DOI: 10.1145/130385.130401

Burton HV and Deierlein GG (2018) Integrating visual damage

simulation, virtual inspection, and collapse capacity to

evaluate post-earthquake structural safety of buildings.

Earthquake Engineering & Structural Dynamics 47(2):

294–310. DOI: 10.1002/eqe.2951

Burton HV, Sreekumar S, Sharma M, et al. (2017) Estimating

aftershock collapse vulnerability using mainshock intensity,

structural response and physical damage indicators. Structural Safety 68: 85–96. DOI: 10.1016/j.strusafe.2017.05.009

Calama R, Mutke S, Tome J, et al. (2011) Modelling spatial and ´

temporal variability in a zero-inflated variable: the case of

stone pine (pinus pinea l.) cone production. Ecological

Modelling 222(3): 606–618. DOI: 10.1016/j.ecolmodel.

2010.09.020. URL https://www.sciencedirect.com/science/

article/pii/S0304380010004916

Chock G (2007) ATC-20 Post-Earthquake Building Safety

Evaluations Performed after the October 15 , 2006 Hawai ‘ i

Earthquakes, Summary and Recommendations for Improvements (updated).

City and County of San Francisco (2020) BORP Guidelines for Engineers. Available online: https://sfdbi.org/borp (accessed on

5 May 2020).

City of Berkeley (2021) The building occupancy resumption

program (borp). Available online: https://www.

cityofberkeley.info/BORP/ (accessed on 27 October 2021).

Computers and Structures, Inc (2021) SAP2000 23.0.0. Available

online: https://www.csiamerica.com/ (accessed on 2 December 2020).

Levine et al. 23

Deierlein G (2021) Performance assessment through nonlinear

time history analysis. Available online: https://apps.peer.

berkeley.edu/grandchallenge/wp-content/uploads/2011/01/

Deierlein-EERI-performance_assessment_through_

nonlinear.pdf (accessed on 21 March 2022).

Diaz J and Joseph MB (2019) Predicting property damage from

tornadoes with zero-inflated neural networks. Weather and

Climate Extremes 25: 100216. DOI: 10.1016/j.wace.2019.

100216. URL https://www.sciencedirect.com/science/

article/pii/S2212094718301385

Earthquake Engineering Research Institute (2011) EERI Special

Earthquake Report: The M 6.3 Christchurch. New Zealand:

Earthquake of February 22, 2011 (May). URL https://www.

eeri.org/site/images/eeri_newsletter/2011_pdf/EERI_

NewZealand_EQRpt_web.pdf

Forrester AIJ, Sóbester A and Keane AJ (2008) Engineering

Design via Surrogate Modelling: A Practical Guide. United

Kingdom: John Wiley & Sons, Ltd. DOI: 10.1002/

9780470770801.ch1

Glorot X, Bordes A and Bengio Y (2011) Deep sparse rectifier

neural networks. Journal of Machine Learning Research 15:

315–323.

Goulet CA, Haselton CB, Mitrani-Reiser J, et al. (2007) Evaluation of the seismic performance of a code-conforming

reinforced-concrete frame building - from seismic hazard

to collapse safety and economic losses. Earthquake Engineering & Structural Dynamics 36(13): 1973–1997. DOI:

10.1002/eqe.694

Hoskere V, Narazaki Y, Hoang T, et al. (2017) Vision-based

structural inspection using multiscale deep convolutional

neural networks. In: 3rd Huixian international forum on

earthquake engineering for young researchers, Qingdao,

China, July 22-25, 2018. URL http://arxiv.org/abs/1805.

01055

Hoskere V, Narazaki Y, Hoang TA, et al. (2018) Towards automated post-earthquake inspections with deep learning-based

condition-aware models. In: The 7th world conference on

structural control and monitoring. 10. URL http://arxiv.org/

abs/1809.09195

Hoskere V, Narazaki Y, Hoang TA, et al. (2020) MaDnet: multitask semantic segmentation of multiple types of structural

materials and damage in images of civil infrastructure.

Journal of Civil Structural Health Monitoring 10: 757–773.

DOI: 10.1007/s13349-020-00409-0

Hoskere V, Narazaki Y and Spencer BF (2022) Physics-based

graphics models in 3d synthetic environments as autonomous vision-based inspection testbeds. Sensors 22(2): 532.

DOI: 10.3390/s22020532. URL https://www.mdpi.com/

1424-8220/22/2/532

Hüthwohl P, Lu R and Brilakis I (2019) Multi-classifier for reinforced concrete bridge defects. Automation in Construction 105: 102824. DOI: 10.1016/j.autcon.2019.04.019

Koch C, Paal SG, Rashidi A, et al. (2014) Achievements and

challenges in machine vision-based inspection of large

concrete structures. Advances in Structural Engineering

17(3): 303–318. DOI: 10.1260/1369-4332.17.3.303

Kubo T and Penzien J (1976) Time and Frequency Domain

Analyses of Three-Dimensional Ground Motions, San Fernando Earthquake. Technical report, Earthquake Engineering Research Center. Berkeley: University of California.

Kwon J (2016) Strength, Stiffness, and Damage of Reinforced

Concrete Buildings Subjected to Seismic Motions. PhD

Thesis, University of Texas at Austin, Austin, TX, USA.

Levine NM, Narazaki Y, Billie F and Spencer J (in review) Development of a building information model-guided post-earthquake

building inspection framework using 3d synthetic environments.

In: Earthquake Engineering and Engineering Vibration.

Levine NM and Spencer BF (2022) Post-earthquake building

evaluation using uavs: a bim-based digital twin framework.

Sensors 22: 873(3). DOI: 10.3390/s22030873. URL https://

www.mdpi.com/1424-8220/22/3/873

Liu PL and Der Kiureghian A (1986) Multivariate distribution

models with prescribed marginals and covariances. Probabilistic Engineering Mechanics 1(2): 105–112. DOI: 10.

1016/0266-8920(86)90033-0. URL https://www.

sciencedirect.com/science/article/pii/0266892086900330

Lorenz TR (2014) 20 years after northridge earthquake be ready

with glendale’s back to business program. Available online:

https://www.glendaleca.gov/Home/Components/News/

News/8/2129?arch=1 (accessed on 27 October 2021).

Miranda E and Aslani H (2003) Probabilistic response assessment

for building-specific loss estimation. Technical report, Pacific

earthquake engineering research center. URL https://peer.

berkeley.edu/sites/default/files/0303_e._miranda_h._aslani.pdf

Mitrani-Reiser J (2007) An ounce of prevention: probabilistic loss

estimation for performance-based earthquake engineering.

ProQuest Dissertations and Theses.

Mitrani-Reiser J, Wu S and Beck JL (2016) Virtual Inspector and its

application to immediate pre-event and post-event earthquake

loss and safety assessment of buildings. Natural Hazards

81(3): 1861–1878. DOI: 10.1007/s11069-016-2159-6

Moehle J and Deierlein GG (2004) A framework methodology for

performance-based earthquake engineering. In: 13th world

conference on earthquake engineering, Vancouver, BC,

Canada, August 1-6, 2004.

Morgenthal G and Hallermann N (2014) Quality assessment of

unmanned aerial vehicle (uav) based visual inspection of

structures. Advances in Structural Engineering 17(3):

289–302. DOI: 10.1260/1369-4332.17.3.289

Narazaki Y, Hoskere V, Hoang TA, et al. (2020) Vision-based

automated bridge component recognition with high-level

scene consistency. Computer-Aided Civil and Infrastructure Engineering 35(5): 465–482. DOI: 10.1111/mice.12505

Narazaki Y, Hoskere V, Yoshida K, et al. (2021) Synthetic environments for vision-based structural condition assessment

of Japanese high-speed railway viaducts. Mechanical Systems and Signal Processing 160: 107850. DOI: 10.1016/j.

24 Advances in Structural Engineering 0(0)

ymssp.2021.107850. URL https://www.sciencedirect.com/

science/article/pii/S0888327021002454

NIST (2017) Guidelines for Nonlinear Structural Analysis for

Design of Buildings: Part iib – Reinforced Concrete Moment

Frames. Technical report. National Institute of Standards and

Technology. DOI: 10.6028/NIST.GCR.17-917-46v3. URL

https://nvlpubs.nist.gov/nistpubs/gcr/2017/NIST.GCR.17-

917-46v3.pdf

Paal SG, Jeon JS, Brilakis I, et al. (2015) Automated damage index

estimation of reinforced concrete columns for post-earthquake

evaluations. Journal of Structural Engineering 141(9):

04014228. DOI: 10.1061/(ASCE)ST.1943-541X.0001200.

Pan X and Yang TY (2020) Postdisaster image-based damage

detection and repair cost estimation of reinforced concrete

buildings using dual convolutional neural networks. Computer-Aided Civil and Infrastructure Engineering 35(5):

495–510. DOI: 10.1111/mice.12549

Paszke A, Gross S, Massa F, et al. (2019) PyTorch: An Imperative

Style, High-Performance Deep Learning Library. Red Hook,

NY, USA: Curran Associates Inc.

Pedregosa F, Varoquaux G, Gramfort A, et al. (2011) Scikit-learn:

machine learning in Python. Journal of Machine Learning

Research 12: 2825–2830.

Porter KA (2003) An overview of PEER’s performance-based

earthquake engineering methodology. 9th International

Conference on Applications of Statistics and Probability in

Civil Engineering 273(1995): 973–980.

Reuland Y, Lestuzzi P and Smith IF (2019) Measurement-based

support for post-earthquake assessment of buildings.

Structure and Infrastructure Engineering 15(5): 647–662.

DOI: 10.1080/15732479.2019.1569071

Spencer BF, Hoskere V and Narazaki Y (2019) Advances in

computer vision-based civil infrastructure inspection and

monitoring. Engineering 5(2): 199–222. DOI: 10.1016/j.

eng.2018.11.030. URL https://www.sciencedirect.com/

science/article/pii/S2095809918308130

SPUR (2012) Safe enough to stay. Technical report, SPUR. URL

https://www.spur.org/sites/default/files/2013-09/SPUR_

Safe_Enough_to_Stay.pdf

Torch Contributors (2019) PyTorch documentation. Available

online: https://pytorch.org/docs/1.11/ (accessed on 15 June

2021).

Vamvatsikos D and Cornell CA (2002) Incremental dynamic

analysis. Earthquake Engineering & Structural Dynamics

31: 491–514. DOI: 10.1002/eqe.141

Virtanen P, Gommers R, Oliphant TE, et al. (2020) SciPy 1.0:

fundamental algorithms for scientific computing in Python.

Nature Methods 17: 261–272. DOI: 10.1038/s41592-019-

0686-2

Wang Y, Cui L, Zhang C, et al. (2022) A two-stage seismic

damage assessment method for small, dense, and imbalanced

buildings in remote sensing images. Remote Sensing 14(4).

1012. DOI: 10.3390/rs14041012. URL https://www.mdpi.

com/2072-4292/14/4/1012

Wang Z, Xu G, Ding Y, et al. (2020) A vision-based active

learning convolutional neural network model for concrete

surface crack detection. Advances in Structural Engineering

23(13): 2952–2964. DOI: 10.1177/1369433220924792

Xu Y, Wei S, Bao Y, et al. (2019) Automatic seismic damage

identification of reinforced concrete columns from images by

a region-based deep convolutional neural network. Structural Control and Health Monitoring 26(3): e2313. DOI: 10.

1002/stc.2313. URL https://onlinelibrary.wiley.com/doi/abs/

10.1002/stc.2313.E2313STC-18-0081.R1

Yazgan U and Dazio A (2012) Post-earthquake damage assessment

using residual displacements. Earthquake Engineering &

Structural Dynamics 41: 1257–1276. DOI: 10.1002/eqe.1184

Yeum CM, Dyke SJ and Ramirez J (2018) Visual data classification

in post-event building reconnaissance. Engineering Structures

155: 16–24. DOI: 10.1016/j.engstruct.2017.10.057

Zhang Y and Burton HV (2019) Pattern recognition approach to

assess the residual structural capacity of damaged tall

buildings. Structural Safety 78(2018): 12–22. DOI: 10.1016/

j.strusafe.2018.12.004

Zhang Y, Burton HV, Sun H, et al. (2018) A machine learning

framework for assessing post-earthquake structural safety.

Structural Safety 72: 1–16. DOI: 10.1016/j.strusafe.2017.12.

001

Levine et al. 25