Assessment of Longstanding Effects of Fly Ash and Silica Fume on the Compressive Strength of Concrete Using Extreme Learning Machine and Artificial Neural Network

Compressive Strength (CS) is an important mechanical feature of concrete taken as an essential factor in construction. The current study has investigated the effect of fly ash and silica fume replacement content on the strength of concrete through Artificial Neural Networks (ANNs) and Extreme Learning Machine (ELM). In this study, different ratio of fly ash with (out) extra quantity of silica fume have been tested. Water cement (w/c) ratio varies during the test. Eight input parameters including Total Cementitious Material (TCM), Silica Fume (SF) replacement ratio, coarse aggregate (ca), fly ash (FA) replacement ratio, Sewage Sludge Ash (ssa) as combination of cement and fine aggregate replacement, water cement ratio, High Ratio Water Reducing Agent (HRWRA) and Age of Samples (AS) and one output parameter as the CS of concrete have been investigated through ANN and ELM. Up to now, numerous experimental studies have been used to analyze the compressive strength of concrete while retrofitted with fly ash or silica fume, however, the novelty of this study is in its use of AI models (ELM, ANN). The models have been developed and their outcomes were compared through six statistical indicators (MAE, RMSE, RRMSE, WI, RMAE and R). Subsequently, both methods were shown as reliable tools for assessing the influence of cementitious material on compressive strength of concrete, however, ANN remarkably was better than ELM. As a result, FA showed less contribution to the strength of concrete at short times, but much at later ages. As a result, the enhanced influence of low amount of SF on CS was not significant. Adding fly ash has reduced the compressive strength in short term, but increased the compressive strength in longterm. Adding silica fume raises the strength in short term, but decreases the strength in long term. 50 © 2021 Journal of Advanced Engineering and Computation (JAEC) VOLUME: 5 | ISSUE: 1 | 2021 | June

Abstract. Compressive Strength (CS) is an important mechanical feature of concrete taken as an essential factor in construction. The current study has investigated the effect of fly ash and silica fume replacement content on the strength of concrete through Artificial Neural Networks (ANNs) and Extreme Learning Machine (ELM). In this study, different ratio of fly ash with (out) extra quantity of silica fume have been tested. Water cement (w/c) ratio varies during the test. Eight input parameters including Total Cementitious Material (TCM), Silica Fume (SF) replacement ratio, coarse aggregate (ca), fly ash (FA) replacement ratio, Sewage Sludge Ash (ssa) as combination of cement and fine aggregate replacement, water cement ratio, High Ratio Water Reducing Agent (HRWRA) and Age of Samples (AS) and one output parameter as the CS of concrete have been investigated through ANN and ELM. Up to now, numerous experimental studies have been used to analyze the compressive strength of concrete while retrofitted with fly ash or silica fume, however, the novelty of this study is in its use of AI models (ELM, ANN). The models have been developed and their outcomes were compared through six statistical indicators (MAE, RMSE, RRMSE, WI, RMAE and R 2 ). Subsequently, both methods were shown as reliable tools for assessing the influence of cementitious material on compressive strength of concrete, however, ANN remarkably was better than ELM. As a result, FA showed less contribution to the strength of concrete at short times, but much at later ages. As a result, the enhanced influence of low amount of SF on CS was not significant. Adding fly ash has reduced the compressive strength in short term, but increased the compressive strength in longterm. Adding silica fume raises the strength in short term, but decreases the strength in long term.

Introduction
There are some demerits of cement production, such as using energy, high raw materials and heat. The production of cement makes air pollution by releasing gaseous emissions and huge solid waste materials. Cement manufacturing is complicated and high cost requiring high different amount and properties of materials, fuel sources such as fuel oil, tiers, natural gas, petroleum coke, coal, and pyro-processing techniques, such as preheating, recirculation and wet and dry kiln. Though, the cement production industry has made essential advances in decreasing CO2 emissions in its procedures, more improvement is confined, since CO2 production is inherent to the main processing of calcinating limestone [1]. Respectively, using aggregate could bring many benefits as low cost, Eco-friendly, Versatile and durable. Also, aggregate is significant for strength, dimensional stability, thermal -elastic properties of concrete and volume stability. Cement is more likely to be affected by shrinkage, however, aggregates could reduce the shrinkage level and prevent cracking. The compressive aggregate strength is another significant parameter in selection of aggregate. Comparing the normal concrete strength with the strength of concrete exposed by aggregates, the later sounds stronger than the conventional concrete. Using aggregates in concrete assist the conserving of raw materials, reduction of carbon dioxide emissions and finally lower environmental pollution beside bringing more sustainability in construction [2]. Among all, silica fume (SF) and fly ash are the two supplementary cementitious aggregates with pozzolanic characteristics (Demirboga et al., 2001; Bilodeau and Malhotra, 2000) carrying economical, ecological and technical benefits [3,4]. SF is a byproduct of producing ferrosilicon alloys or silicon metal due to is its physical and chemical properties as a very reactive pozzolan (Khedr and AbouZaid 1(94)) [5]. By using silica fume in concrete in Canada, it was considered as a cement replacement in normal strength concrete in order to gain a favorite 28-day CS [ Fly ash is also a waste material derived from the combustion of pulverized coal in electricity generating plants Fly ash particles are generally solid glassy spheres, from grey to tan to reddish brown and range in size from 2 µm to 10 µm. Fly ash is the unburned residue carried away from the burning zone in the boiler by the flue gases and then gathered by electrostatic or mechanical separators [31][32][33]. It comprises mostly aluminum oxide (Al2O3), iron oxide (Fe2O3) and silicon dioxide (SiO2) [34]. Despite the common use of fly ash over 50 years, there are confinements over its usage in concrete due to unknown characteristics of fly ash in concrete [35,36] which is highly based on the type of applied coal, temperature of fire, air fuel ratio, the combustion condition and collector setup [37][38][39][40]. In this case, some generalized processing is made in surface area, morphology, density, distribution of particle size, hydraulic conductivity or permeability of fly ash [41]. Gen- erally, fly ash comprises two classes as fly ash Class C and fly ash Class F. Fly ash from class C typically is made from the combustion of subbituminous coal or younger lignite, however, fly ash class F is made from the burning of older anthracite and harder bituminous coal. The major alteration between the two classes is the amount of alumina, silica, iron, and calcium content. Class F fly ash consist of less than 10% CaO and minimum Al2O3 + Fe2O3 + SiO2 content of 70%, whereas Class C fly ash consist of more than 20% CaO and minimum Fe2O3 + Al2O3 + SiO2 content of 50% [32,42]. Kearsley and Wainwright (2001) have investigated the influence of fly ash on CS of concrete, finding that CS is highly depending on the fly ash dry density [43]. It was also found that the high replacement ratio of fly ash instead of cement couldn't significantly impact the concrete's strength in long term (Kearsley and Wainwright, 2001). In another study by Behnood and Golafshani (2018), ANN is used with Multi-Objective Grey Wolves (HANNMOGW) method to predict the CS of silica fume concrete, resulting a linear increment in the CS of silica fume concrete when the silica fume to binder ratio is raised from 0 to approximate 30%. Also, the CS of silica fume concrete is significantly affected by the maximum aggregate size [ Table 3 shows some samples of the methods that was previously applied to estimate the CS of various concrete types.
ANN (as an artificial intelligence) is commonly applied for predicting different properties of concrete mixtures by solving very rigorous problems through the interconnected computing elements. ANN process resembles the brain network including many simple computational parameters arranged in layers. A concrete designed by ANN is expected to have optimal water and cement that should bring more durability and likely better ecological and economic impacts [63]. Neural networks (NN) could "learn" and "correlate" the large datasets gained from simulations and tests. A trained NN performs as an analytical tool for the qualified prognoses of real outcomes, showing highly accurate scores in their predictions. Asteris  . The current study, by use of AI models has attempted to provide a precise analysis in predicting the compres-sive strength of concrete while exposed by fly ash and silica fume. Many studies have analyzed the addition of aggregates to concrete, however, using AI models is rarely seen. Accordingly, this research has attempted to provide a precise and accurate analysis to the case. This study has used ANN and ELM align with the objective of this study in terms of using AI models to predict the compressive behavior of concrete while adding aggregates. In the following, the models have been developed and their results were illustrated in diagrams. Comparing this study with the original one that data was derived from is in methodology. While the original paper only used one AI model (neural network), the current study has used two AI models for the same analysis of adding fly ash and silica fume to concrete.

Problem Statement and Objective
Regarding the concrete content (chemical admixtures, water, supplementary cementitious materials, fine and coarse aggregates, cement and fibers), one big problem is to achieve and maintain the specified CS in concrete design. Due to the low knowledge in terms of the tests between concrete strength and its constituents, finding the reliable and novel machine learning methods and soft computing to gain those relationships has been provoked [75]. Having to try all the different quantities and testing each one's performance takes enormous time, effort, cost and might with less accuracy (experimental error). To capture a 28-day design CS with proper workability, the technical personnel have to highly attempt for several mix percentages that takes long time, raising the wastage of material and cost of concrete production. Significantly, in case of concrete test failures after so long waiting, there might be highly difficult to adjust another test as before. On the other hand, for improving the strength and sustainability of concrete, using aggregates, such as silica fume, fly ash and slag are highly applied from the last few decades. In this case, there are many models for prediction and estimating the strength of concrete well before 28 days as Abrahams Law [76]. However, in reality, the relationship between the strength of concrete and its constituent's material is highly nonlinear and Abrahams Law model is failed satisfactorily to compute this complicated relationship and falling short to generalize the unseen data. Unlike Abrahams Law, AI was proved as reliable and accurate tool in learning rigorous patterns, besides saving time and money. Among AI algorithms, ANN is broadly suggested to predict the strength of concrete through the use of back propagation (BP) network. Adding that BP fails from the local minima that is led to the instability in the developed model. The convergence to local minima is due to the optimization objective of ANN that is naturally multimodal. It is due to the fact that various training data sets generate various models affecting the capability of that model in generalizing to previously unseen data. It could be a confinement to ANN reported in the related studies particularly in medical estimation and predictions [77]. Though there might be a solution to this by aggregating the outputs of few models developed from the training data as "boosting", "bagging" or stacking predictors, this brings more computations and some uncertainty [78,79]. The use of nontuned machine learning model e.g. ELM could bring high progress in multidisciplinary of engineering and science fields in past few years. This is due to its superiority on standard ANN algorithms, such as the randomly initiated hidden neurons without the need for iterative tuning process for free parameters or connections between hidden and output layers. ELM as a relatively new learning method for feedforward NNs, unlike conventional NNs, the hidden biases and input weights are randomly initialized and remain fixed across the learning process plus the analytically determination of output weights, all bring to ELM a good generalized performance and fast learning speed [74]. Ultimately, ELM is highly proficient in gaining a global optimum while following global approximating capability of single layer feed-forward network. Also, ELM is superior because of its generalized performance in estimating problems in various fields than the conventional models as ANNs or SVMs. On the other hand, ANN is better than the multiple regression ones, particularly in decreasing the scatter of predictions in concrete mixture. ANN has been widely used in predicting the self-compacting concrete (SCC), CS of conventional concrete and high performance concrete (HPC). Then, ANN is able to predict the CS of flowable concrete in an appropriate admixture of concrete.

Methodology
The data of this research was originally obtained from the study of "Appraisal of long-term effects of fly ash and silica fume on CS of concrete by  Table 4 shows the input and output parameters used in this investigation. including silica fume replacement ratio (SF), total cementitious material (TCM), coarse aggregate (ca), water content (W), fly ash replacement ratio (FA), fine aggregate (ssa), age of samples (AS), high rate water reducing agent (HRWRA) and one output parameter as compressive strength of concrete (fc).

Statistical data
Tab. 4: Input -output parameters in database.

Inputs
Parameters description input 1 FA (%) input 2 SF (%)  (1): x i = nodal variables in previous layer n = all nodal variable' numbers received from the previous layer w ij & b j = the weights and biases of network in the current layer In order to decrease the net values' diversity, an activation function is used to the net, then y as output signal was reported [88]. The tangent hyperbolic function is regarded as the highest reliable activation performances as Eq. (2): This process is performed on each layer of MLP until the output signals of the last layer (predicted values) are reported. Later, the differential of the observed variables (error) and predicted ones have been computed. Ultimately, one optimized algorithm is used to decline the error variable by modifying the assigned biases and weights through MLP. This training procedure can be performed through diverse algorithms, however, due to the high speed of convergence and high accuracy of backpropagation algorithms, ANN is more preferable in this study [89]. Fig. 1 indicates the scheme of an ANN.

Extreme Learning Machine (ELM)
Huang et al. proposed ELM to tune the single layer feed forward NN (SLFNN) architectures [90]. ELM is derived from the notion that an 9 verse algorithms, however, due to the high speed of convergence and high accuracy of ation algorithms, ANN is more preferable in this study [89]. Fig. 1 indicates the scheme of .
(3) G = the activation performance (all the NN based activation functions could be applied here as well) T = weight vector connecting i th input neurons to hidden neuron Supposing that an SLFNN with L hidden neurons and activation function G could compute the targets (t j ) with 0 error, i.e.
Also, this N equations could be compactly written as: in which: and, The output weights are gained if the minimal differential between the right side (target variables) and the left side (predicted variables) of Eq. (5) is occurred e.g. min Hβ-T . Based on the mathematical theories, it was proved that when the output weight is as follows, the minimal error between the target variables and predicted variables occurs: As noted, no optimization process is involved in ELM, then the human inferences and the training time are reduced efficiently. The components of ELM are shown in Fig. 2.

Results and Discussions
For evaluating and testing the models' performance used in this research, 70% of the data is randomly separated and assigned to training section, and 30% data is used to testing phase. As ted, no optimization process is involved in ELM, then the human inferences and the training re reduced efficiently. The components of ELM are shown in Fig. 2. a measurement parameter, statistical model performance indicators comprising mean absolute error (MAE ), root mean square (RMSE ), relative root mean squared error (RRMSE ), Willmott's index (WI), relative mean absolute error (RMAE ) and determination coefficient (R 2 ) were applied. For comparing the models, all codes are written in MATLAB without using external toolbox or compiler. Afterwards, codes were computed a processor (Intel(R) Core (TM) i5-8250U CPU 1.60 GHz 1.80 GHz and 8.0 GB RAM).

1) ANN Development
The performance of ANN obviously is based on the layout and model's architecture, say the neurons' number and the hidden layers' number used in each layer. An error and trail processing was performed to gain the desired performance in order to find the appropriate ANN architecture. Altered architectures with different neurons and hidden layers have been made while each model was run 3 times with 800 epochs. This is because in AI models generally and ANN particularly, data are selected randomly in test and train phases (70% train, 30% test), so probably the neurons that are going to predict the output could provide a good prediction (or not). For reducing the probability of randomness and avoiding the adverse effect on our output, data is run three times, then the average of three times is selected as the output. In Tabs. 5 and 6, the RMSE values show the average of the data after three runs. In this research, Levenberg-Marquardt (LMA) was applied to define the bias and weight of ANN neurons due to its fast performance in achieving optimal performance among BP algorithms. Finally, the mean value of RMSE was achieved to indicate the performance of models and their comparison. Table 5 shows the RMSE variables (single-layer) of ANN in trial and error processing that was modified for each layer and neurons number. Figure 3 shows the RMSE output of ANN with the neurons 1, 2, 3, . . . 10 as single-layer in testing and training phases. According to Tab. 6 and Fig. 4, there are 6 neurons in the first layer and 4 neurons in the second layer with the RMES values of 1.534524 and 2.208552 in training and testing phase, respectively. Since there weren't good results in the RMSE values of the single-layer, then the second layer was used and provided good results. Though few models with altered combinations of layers and neurons gain lower RMSE variable in training phase, they are not able to gain this performance in testing phase. Figure 4 shows the both layers (single and two-layer) in ANN model while the first layer has 6 neurons and the second layer has 4 neurons. It is noted that when the RMSE of test was increasing, it is not good for accurate prediction, so adding neuron was stopped in testing phase and the least amount was taken as the best result. Thereafter, adding neuron in the second layer was stopped and point x (Fig. 4) with 6 neurons in the first layer and 4 neurons in the second layer is accepted as the best RMSE point and the best ANN performance in the second layer. As a result, ANN couldn't represent good performance in the first layer (single-layer), however, it showed its best performance in the second layer (two-layer). Considering the innovation of using ANN, data was initially analyzed in the first layer and the output was obtained (Fig.  3). Later, the same process was performed in the second layer (Fig. 4), showing the best performance of ANN in the second layer (point x).
wo-layer) in ANN model while first layer has 6 neurons and the second layer has 4 that when the RMSE of test was increasing, it is not good for accurate prediction, was stopped in testing phase and the least amount was taken as the best result.  ANN Results: Following Tab. 5, the monolayer arrangement of neurons is tested and the results is depicted in Fig. 3 in training and testing phases. Table 6 shows the two-layer arrangement along the training and testing results. Figure  4 shows the best (lowest) RMSE in testing and training phases.   Results of the AI models for concrete data set in testing phase

Error Distribution Observed Error Values
Comparison of observed and predicted compressive strength for ANN Figure 6. ANN model testing outputs

ELM Development
Unlike ANN, ELM deals with single layer feed forward neural network (SLFNN) architectures. There is no need to define the hidden layers' number, then SLFNN was determined and leads to the network architecture parameter specifying. A trial and error process (such as ANN) has been carried out to specify the number of ELM neurons. Subsequently, ELM performance is tested by the RMSE index

2) ELM Development
Unlike ANN, ELM deals with single layer feed forward neural network (SLFNN) architectures. There is no need to define the hidden layers' number, then SLFNN was determined and leads to the network architecture parameter specifying. A trial and error process (such as ANN) has been carried out to specify the number of ELM neurons. Subsequently, ELM performance is tested by the RMSE index during the training and testing steps in this trial and error process. Table 7 shows the RMSE of ELM model in trial and error process with their corresponding variables as one layer. By adding the neurons' amount, it is not good for accurate prediction, so 67 neurons were stabled. Since ELM is involved with only one layer, considering all the RMSE values in this layer, the architecture with 67 neurons and the RMSE value of 4.6025 in testing phase is accepted as the best performance of ELM. Figure 7 shows the RMSE output of ELM with the neurons of 50, 52, 54,. . . 80 as one hidden layer in testing and training phases. Considering the innovation of ELM, based on different neurons, output was obtained (Fig. 7), showing that the architecture of 67 was the best performance of single layer ELM. In terms of cost for both models, ANN designing takes more cost and accuracy due to its layers than ELM with one layer.     are compared and since the value of ANN is near to 1, then the best performance metrics belong to ANN in approximating the CS of concrete in adding silica fume and fly ash. As a result, ANN represented superior performance in this study.

Conclusion
Predicting the CS of concrete is an important challenge in construction industry. In reality, calculating the concrete strength in traditional testing method takes time, cost, error in percentages and lack of accuracy. Accordingly, the CS of concrete has been analyzed while adding silica fume and fly ash through AI algorithms as ANN and ELM [13]. Collecting data set was from the appraisal of long-term influences of silica fume and fly ash on the CS of concrete by NNs including 144 various concrete mixtures. Water cement (w/c) ratio was differed during the test comprising the high and low FA content and with(out) extra small SF content. The concrete of samples was cured for 3, 7, 28, 56 and 180 days. To achieve the best performance of the models, a trial and error method has been performed to gain the optimum values of free parameters. Due to inadequate outcome in the 1 th layer with 6 neurons in ANN, the 2 nd layer was used comprising 4 neurons (second layer), then the best performance of ANN is observed in the second layer with 4 neurons. Alternatively, due to SLFNN architecture of ELM, there was only one hidden layer in ELM and the best architecture was seen with 67 neurons as the best ELM performance. After analyzing the RMSE and R 2 (RSQR) values of both models derived from the training and testing phases of the two models, it resulted that ANN could remarkably show better performance than ELM. Since the best RMSE should be equal or near to 0, then the RMSE of ANN (2.2086) is close to 0, however, the RMSE of ELM (4.6025) is far from 0. As a result, ANN could perform better than ELM in estimating and predicting the CS of concrete while adding silica fume and fly ash. On the other hands, by comparing the two values of R 2 (RSQR) in ANN and ELM, the R 2 value of ANN (0.9905) and ELM (0.9616) should be equal or close to 1 (as the best R 2 value) in order to be nominated as an accurate model for predicting the CS of concrete in this study. Therefore, while the R 2 value of ELM is far from 1, the R 2 value of ANN is close to 1, then ANN is accounted as the best model in predicting and estimating the CS of concrete while adding silica fume and fly ash. Consequently, ANN was superior to ELM in accurately approximating the CS of concrete. Also, it was found that fly ash had less contribution to the compressive strength of concrete at early ages, however, had great contribution at later ages. It can also be concluded that the enhancement effect of low content of SF on compressive strength was not significant. On the other hand, adding fly ash has decreased the early compressive strength, but raised the compressive strength in the long-term. Adding silica fume raises the strength in short term, but decreases the strength in long term. Comparing the results of this study with the original one, going through the R 2 value of the original one (0.9990) in testing phase and the R 2 value of this study is 0.9905 in testing in ANN model. Therefore, both studies could represent the same accuracy with the least differential percentages. [23] Behnood, A., & Ziari, H. (2008). Effects of silica fume addition and water to cement ratio on the properties of high-strength concrete after exposure to high temperatures. Cement and Concrete Composites, 30 (2), 106-112.
[26] Aköz, F., et al. (1999). Effects of raised temperature of sulfate solutions on the sulfate resistance of mortars with and without silica fume. Cement and Concrete Research, 29 (4), 537-544.  Majid KHORAMI is a Ph.D candidate in Politecnic University of Valencia-Spain. Currently, he is working as an associate professor in the Faculty of Architecture and Urbanism, UTE University in Ecuador with more than a decade of research and academic teaching experience. His area of research is Ultra-High Performance Fibre Concrete (UHPFRC), applying artificial intelligence, concrete technology and sustainable construction. He has authored/co-authored over 50 papers published in well-known international journals and reviewer over 5 journals.