Example of a 2D-SFS

The following matrix contains the 2D-SFS of 50,000 simulated sites from two populations, with a sample size of 2 diploids for pop1 and 3 diploids for pop2. Hence, we have 7 rows and 5 columns. Each entry (i,j) contains the number of SNPs with a derived allele frequency of i in pop1, and j in pop2. The sum of the entries in this matrix is the total number of sites, which in this case is 50,000. This matrix can be visualy represented by a heatmap, as in the figure below.

Exercise 1.2.2.
Based on the above matrix with the 2D SFS, answer the following questions.

• How many sites have 1 heterozygote individual in pop1 and 1 heterozygote individual in pop2?

• How many sites have a derived allele frequency of 4 in pop1 and a derived allele frequency of 4 in pop2?

• Can you identify the entry corresponding to the sites where all individuals are heterozygote?

Exercise 1.2.3.
Based on the above matrix with the 2D SFS, obtain the marginal 1D SFS.

2.1 Input files: DATA

The input files with the observed SFS are in the folder: “./FscInputFiles/”:

• 2PopDiv20Mb_jointDAFpop1_0.obs: 2D observed SFS for the model with two populations.

Exercise 2.1. Have a look at the input files using a text editor (e.g. RStudio editor, gedit, TextPad, Notepad). You can read them with R using the read.table() function.

# set the working directory
setwd("/home/popgen/workshop_materials/22_fastsimcoal")
# read the observed SFS
# 2D from two populations with different effective sizes that diverged some time ago
pop2d <- read.table("./FscInputFiles/2PopDiv20Mb_jointDAFpop1_0.obs", skip=1, stringsAsFactors = F, header=T, row.names = 1)
# 
head(pop2d)

The observed derived SFS is:

Have a look at the observed SFS file. Does the observed SFS include the monomorphic sites?

The entry for the monomorphic sites is the one with a derived frequency of zero in pop1 and in pop2, i.e. the entry [1,1] corresponding to the 1st row and 1st column.

Input files 2.2: DEFINITION OF MODELS

Exercise 2.2.
Have a look at the TPL and EST files for the models shown above. Use a texteditor (e.g. RStudio editor, gedit, TextPad) to have a look at these files.

Each model is defined in a TPL file. The parameters we want to infer are given a parameter name tag (e.g. NPOP, TDIV). Have a look at the TPL file on the folder “FscInputFiles”, corresponding to the model:

• 2PopDiv20Mb.tpl

The search range for each parameter is defined in the EST file. For each parameter we specify the search range using the corresponding parameter name tag. Have a look at the EST file on the folder “FscInputFiles”, corresponding to the two model:

• 2PopDiv20Mb.est

In sum, there are two steps:

1. Define the model with the TPL file, specifying the parameter tags for those parameters we want to infer. NOTE: different parameters need to be given different parameter name tags, and name tags cannot be a sub-string another one (e.g. NPOP1, NPOP1_2 cannot be used because NPOP1 is a sub-string of NPOP1_2). For this reason, we usually use a “$” at the start and end of a tag, e.g. $NPOP1$, $NPOP1_2$.

2. Define the search range for each parameter in the EST file.

For further information on definition of models check the manual and the material given during the lecture about specification of models in fastsimcoal2. NOTE: the TPL and EST files need to have the same filename, but have different extensions: [filename].est and [filename].tpl.

TO DISCUSS
From looking at the TPL and EST file:
– What is the mutation rate assumed?
– Do we need a mutation rate estimate to be able to run fastsimcoal2?
– Can we estimate the mutation rate from the SFS?

2.3. Settings to run fastsimcoal2

To run fastsimcoal2 we need to specify the following options:

• the number of coalescent simulations to approximate the expected SFS (-n option). This should be larger than 100,000. We recommend to use 200,000 to 1,000,000.

• the number of optimization cycles (-L option). This should be at least 20, but values between 50 and 100 are recommended.

• use option -M indicating that we will run the likelihood optimization to infer parameters based on the SFS.

• use option -d for the derived SFS and -m for the MAF SFS.

• use option -0 if you do not have monomorphic sites in your observed SFS. NOTE: in this case you need to fix a parameter (e.g. a split time or an effective size), and all parameters will be relative to that fixed parameter.

• use option -q to have a quite mode, without printing lots of information.

• specify option -C1 such that the likelihood is computed for all entries with at least 1 SNP. If you specify -Cx then all entries with less than x SNPs are pooled together. This option is useful when there are many entries in the observed SFS with few SNPs and with a limited number of SNPS to avoid overfitting.

• specify options for multithread. -c1 -B1 for a single core, -c4 -B4 for 4 cores using 4 batches.

2.4. Run fastsimcoal2 from R

The following scripts will run 1 fastsimcoal2 optimization, starting from a random initial starting value. When dealing with real data one needs to repeat at least 20 to 100 runs (i.e. runs with different initial starting values), and then we select the run that maximizes the likelihood (i.e. with the highest MaxEstLhood).

# load functions to run fastsimcoal2 and to process the output 
source("utilFscFunctions.r")
source("ParFileInterpreter_VS.r")

# create a list that saves all the required settings to run fastsimcoal2
settings <- list()
# path to fastsicoal2 executable file
# in this case the fastsimcoal2 executable is in the folder /home/popgen/software/fsc26_linux64/
settings$pathToFsc <- "/home/popgen/software/fsc26_linux64/"
# DEFINE OBSERVED SFS INPUT FILE
# path to input files (working directory)
# In this case, the input files are in the folder ./FscInputFiles/
settings$pathTo_InputFile <- "./FscInputFiles/"
# DEFINE THE MODEL WITH EST AND TPL FILES 
# tags to the TPL and EST files. Files should be "tag".tpl and "tag".est
# in this case files are 2PopDiv20Mb.tpl and 2PopDiv20Mb.est
settings$TPL_EST_file_tag <- "2PopDiv20Mb"
# number of coalescent simulations
settings$n_coalsims <- 100000
# number of optimization cycles
settings$n_cycles <- 20

# IF YOU WANT TO RUN FASTSIMCOAL2 FROM THE COMMAND LINE
# get the fastsimcoal2 command line
cmd <- get_fsc2_commandline(settings)
cmd
# IF YOU WANT TO RUN FASTSIMCOAL2 BY CALLING IT FROM R
# Only works in Unix systems
# run fastsimcoal2
run_fsc2(settings)

Running fastsimcoal2 should take less than 2min. You will get a print of the optimization procedure, with the likelihood at each iteration. For instance, I obtained the following in one of the runs:

Estimation of parameters by conditional maximization via Brent algorithm (initial lhood = -356267)
Iter 1 Curr best params: 24107 20448 12188 1119 lhood=-138019
Iter 2 Curr best params: 13304 5500 9889 1012 lhood=-137381
Iter 3 Curr best params: 20621 5307 9790 1046 lhood=-137361
…
Iter 20 Curr best params: 20414 5155 9895 1013 lhood=-137360

The likelihood is given in units of log10. Hence, the closer to zero the higher the likelihood. We can see that the likelihood increased from -356267 to -137360 in 20 cycles of the optimization. We can also see that after 1 iteration the parameters were (in same order as in EST file): NPOP1 24107, NPOP2 20448, NANC 12188, TDIV 1119 and at iteration 20 the values were NPOP1 20414, NPOP2 5155, NANC 9895, TDIV 1013.

How to interpret fastsimcoal2 results? Fastsimcoal2 will output several files, which contain important information. The most relevant information for most users is:

• *.bestlhoods : file with the Maximum likelihood parameter estimates and corresponding likelihood. This is what we want – parameter estimates!

• *_DAFpop0.txt : file with the expected SFS obtained with the parameters that maximized the likelihood during optimization. This is needed to visually check the fit of the SFS. Or we can use this to recompute the likelihood of other observed SFS (e.g. without linked sites) given our model.

• *.simparam : file with an example of the settings to run the simulations. It is useful to check this file when you have errors.

Exercise 2.4
Obtain parameter estimates and maximum likelihood.
Follow the code below that will read the output files from fastsimcoal2 to obtain the parameter estimates.

# read the file with the maximum likelihood estimates
maxlhoodEstimates <- read.table(paste(settings$pathTo_InputFile, "/",
                                      settings$TPL_EST_file_tag, "/",
                                      settings$TPL_EST_file_tag, ".bestlhoods",
                                      sep=""), header=T)
# look at the parameter estimates
maxlhoodEstimates

The maxlhoodEstimates contain a matrix where each column corresponds to a parameter name tag and the corresponding parameter estimate. You have the result of 1 run. In the analysis of real data sets we need to run several independent runs (different starting values), and select the parameter estimates of the run attaining the maximum likelihood estimate (MaxEstLhood).

TO DISCUSS:
– What are the parameter estimates of your run?

NOTE: Difference between MaxObsLhood and MaxEstLhood:

MaxObsLhood: is the maximum possible value for the likelihood if there was a perfect fit of the expected to the observed SFS, i.e. if the expected SFS was the relative observed SFS.

MaxEstLhood: is the maximum likelihood estimated according to the model parameters.

The better the fit, the smaller the difference between MaxObsLhood and MaxEstLhood

So, in your case, all participants should have the same MaxObsLhood because all students are looking at the same OBSERVED SFS, but each one of you will have a different MaxEstLhood because you have run different optimization cycles, starting from different initial values.

Exercise 2.5
– Assess the fit of the expected SFS under the model to the observed SFS.

Follow the code below that will read the output files from fastsimcoal2 to obtain the observed and expected SFS.

# Fit of the model expected SFS to the observed SFS

# Tag for the end of the observed SFS file
obsfileend <- "_jointDAFpop1_0"

# Read the observed SFS - SNP counts
obssfs <- read.table(paste(settings$pathTo_InputFile,
                    settings$TPL_EST_file_tag, obsfileend, ".obs",
                    sep=""), skip=1, stringsAsFactors = F, header=T)

# Read the expected SFS - PROBABILITIES
expsfs <- read.table(paste(settings$pathTo_InputFile,
                    settings$TPL_EST_file_tag, "/",
                    settings$TPL_EST_file_tag, obsfileend, ".txt",
                    sep=""), header=T)

# Plot the fit of the 2D SFS, including of the marginal 1D SFS
# the function plot2dSFS is defined in the utilfunctions.r
# you need to give as input the following arguments
#    obsSFS : matrix with observed SFS (counts)
#    expSFS : matrix with expected SFS (probabilities)
#    xtag : string with the label of x-axis
#    ytag : string with the label of y-axis
#    minentry : number with the minimum entry in the SFS (all entries with less than this are pooled together)
plot2dSFS(obsSFS=obssfs, expSFS=expsfs, xtag="Pop2", ytag="Pop1", minentry=1)

# Draw a scenario of the model parameter estimates assuming a generation time of 1 year
# path to the maxL.par file that is a par file with the parameters that maximize the likelihood
path_to_maxL_file <- paste(settings$pathTo_InputFile, settings$TPL_EST_file_tag, "/",settings$TPL_EST_file_tag, "_maxL", sep="")
parFileInterpreter(args=path_to_maxL_file, pop.names=c("Pop1","Pop2"), gentime=1, printPDF=FALSE)

## [1] "./FscInputFiles/2PopDiv20Mb/2PopDiv20Mb_maxL"

Note the difference between the observed SFS and the expected SFS. The observed SFS shows the number of sites with a given frequency (counts, which are integer values), whereas the expected SFS shows the probability of finding a site with a given absolute allele frequency in both populations (i.e. probabilities with values between 0 and 1.0).

DISCUSSION
– Is the method able to infer the correct parameters?
– Does the expected SFS fit the observed SFS?
– Does the SFS contain information about the demographic history of populations?
– Time estimates are given in generations. How to convert to years?