Computational Methods for Tonality-Based Style Analysis of Classical Music Audio Recordings

Computational Methods for Tonality-Based Style Analysis of Classical Music Audio Recordings. Weiß, C. Ph.D. Thesis, Technische Universität Ilmenau, 2016.

Paper doi abstract bibtex

With the tremendously growing impact of digital technology, the ways of accessing music crucially changed. Nowadays, streaming services, download platforms, and private archives provide a large amount of music recordings to listeners. As tools for organizing and browsing such collections, automatic methods have become important. In the area of Music Informa- tion Retrieval, researchers are developing algorithms for analyzing and comparing music data with respect to musical characteristics. One typical application scenario is the classification of music recordings according to categories such as musical genres. In this thesis, we approach such classification problems with the goal of discriminating subgenres within Western classical music. In particular, we focus on typical categories such as historical periods or individual composers. From a musicological point of view, this classi- fication problem relates to the question of musical style, which constitutes a rather ill-defined and abstract concept. Usually, musicologists analyze musical scores in a manual fashion in order to acquire knowledge about style and its determining factors. This thesis contributes with computational methods for realizing such analyses on comprehensive corpora of audio recordings. Though it is hard to extract explicit information such as note events from audio data, the computational analysis of audio recordings might bear great potential for musi- cological research. One reason for this is the limited availability of symbolic scores in high quality. The style analysis experiments presented in this thesis focus on the fields of harmony and tonality. In the first step, we use signal processing techniques for computing chroma representations of the audio data. These semantic “mid-level” representations capture the pitch class content of an audio recording in a robust way and, thus, constitute a suitable starting point for subsequent processing steps. From such chroma representations, we derive measures for quantitatively describing stylistic properties of the music. Since chroma features suppress timbral characteristics to a certain extent, we hope to achieve invariance to timbre and instrumentation for our analysis methods. Inspired by the characteristics of the chroma representations, we model in this thesis specific concepts from music theory and propose algorithms to measure the occurence of certain tonal structures in audio recordings. One of the proposed methods aims at estimating the global key of a piece by considering the particular role of the final chord. Another contribution of this thesis is an automatic method to visualize modulations regarding diatonic scales as well as scale types over the course of a piece. Furthermore, we propose novel techniques for estimating the presence of specific interval and chord types and for measuring more abstract notions such as tonal complexity. In first experiments, we show the features' behavior for individual pieces and discuss their musical meaning. On the basis of these novel types of audio features, we perform comprehensive experiments for analyzing and classifying audio recordings regarding musical style. For this purpose, we apply methods from the field of machine learning. Using unsupervised clustering methods, we investigate the similarity of musical works across composers and composition years. Even though the underlying feature representations may be imprecise and error-prone in some cases, we can observe interesting tendencies that may exhibit some musical meaning when analyzing large databases. For example, we observe an increase of tonal complexity during the 19th and 20th century on the basis of our features. As an essential contribution of this dissertation, we perform automatic classification experiments according to historical periods (“eras”) and composers. We compile two datasets, on which we test common classifiers using both our tonal features and standardized audio features. Despite the vagueness of the task and the complexity of the data, we obtain good results for the classification with respect to historical periods. This indicates that the tonal features proposed in this thesis seem to robustly capture some stylistic properties. In contrast, using standardized timbral features for classification often leads to overfitting to the training data resulting in worse performance. Comparing different types of tonal features revealed that features relating to interval types, tonal complexity, and chord progressions are useful for classifying audio recordings with respect to musical style. This seems to validate the hypothesis that tonal characteristics can be discriminative for style analysis and that we can measure such characteristics directly from audio recordings. In summary, the interplay between musicology and audio signal processing can be very promising. When applied to a specific example, we have to be careful with the results of computational methods, which, of course, cannot compete with the experienced judgement of a musicologist. For analyzing comprehensive corpora, however, computer-assisted techniques provide interesting opportunities to recognize fundamental trends and to verify hypotheses.

@PhDThesis{        wei2016-computational,
    author       = {Wei{\ss}, Christof},
    year         = {2016},
    title        = {Computational Methods for Tonality-Based Style Analysis
                   of Classical Music Audio Recordings},
    abstract     = {With the tremendously growing impact of digital
                   technology, the ways of accessing music crucially changed.
                   Nowadays, streaming services, download platforms, and
                   private archives provide a large amount of music
                   recordings to listeners. As tools for organizing and
                   browsing such collections, automatic methods have become
                   important. In the area of Music Informa- tion Retrieval,
                   researchers are developing algorithms for analyzing and
                   comparing music data with respect to musical
                   characteristics. One typical application scenario is the
                   classification of music recordings according to categories
                   such as musical genres. In this thesis, we approach such
                   classification problems with the goal of discriminating
                   subgenres within Western classical music. In particular,
                   we focus on typical categories such as historical periods
                   or individual composers. From a musicological point of
                   view, this classi- fication problem relates to the
                   question of musical style, which constitutes a rather
                   ill-defined and abstract concept. Usually, musicologists
                   analyze musical scores in a manual fashion in order to
                   acquire knowledge about style and its determining factors.
                   This thesis contributes with computational methods for
                   realizing such analyses on comprehensive corpora of audio
                   recordings. Though it is hard to extract explicit
                   information such as note events from audio data, the
                   computational analysis of audio recordings might bear
                   great potential for musi- cological research. One reason
                   for this is the limited availability of symbolic scores in
                   high quality. The style analysis experiments presented in
                   this thesis focus on the fields of harmony and tonality.
                   In the first step, we use signal processing techniques for
                   computing chroma representations of the audio data. These
                   semantic “mid-level” representations capture the pitch
                   class content of an audio recording in a robust way and,
                   thus, constitute a suitable starting point for subsequent
                   processing steps. From such chroma representations, we
                   derive measures for quantitatively describing stylistic
                   properties of the music. Since chroma features suppress
                   timbral characteristics to a certain extent, we hope to
                   achieve invariance to timbre and instrumentation for our
                   analysis methods. Inspired by the characteristics of the
                   chroma representations, we model in this thesis specific
                   concepts from music theory and propose algorithms to
                   measure the occurence of certain tonal structures in audio
                   recordings. One of the proposed methods aims at estimating
                   the global key of a piece by considering the particular
                   role of the final chord. Another contribution of this
                   thesis is an automatic method to visualize modulations
                   regarding diatonic scales as well as scale types over the
                   course of a piece. Furthermore, we propose novel
                   techniques for estimating the presence of specific
                   interval and chord types and for measuring more abstract
                   notions such as tonal complexity. In first experiments, we
                   show the features' behavior for individual pieces and
                   discuss their musical meaning. On the basis of these novel
                   types of audio features, we perform comprehensive
                   experiments for analyzing and classifying audio recordings
                   regarding musical style. For this purpose, we apply
                   methods from the field of machine learning. Using
                   unsupervised clustering methods, we investigate the
                   similarity of musical works across composers and
                   composition years. Even though the underlying feature
                   representations may be imprecise and error-prone in some
                   cases, we can observe interesting tendencies that may
                   exhibit some musical meaning when analyzing large
                   databases. For example, we observe an increase of tonal
                   complexity during the 19th and 20th century on the basis
                   of our features. As an essential contribution of this
                   dissertation, we perform automatic classification
                   experiments according to historical periods (“eras”)
                   and composers. We compile two datasets, on which we test
                   common classifiers using both our tonal features and
                   standardized audio features. Despite the vagueness of the
                   task and the complexity of the data, we obtain good
                   results for the classification with respect to historical
                   periods. This indicates that the tonal features proposed
                   in this thesis seem to robustly capture some stylistic
                   properties. In contrast, using standardized timbral
                   features for classification often leads to overfitting to
                   the training data resulting in worse performance.
                   Comparing different types of tonal features revealed that
                   features relating to interval types, tonal complexity, and
                   chord progressions are useful for classifying audio
                   recordings with respect to musical style. This seems to
                   validate the hypothesis that tonal characteristics can be
                   discriminative for style analysis and that we can measure
                   such characteristics directly from audio recordings. In
                   summary, the interplay between musicology and audio signal
                   processing can be very promising. When applied to a
                   specific example, we have to be careful with the results
                   of computational methods, which, of course, cannot compete
                   with the experienced judgement of a musicologist. For
                   analyzing comprehensive corpora, however,
                   computer-assisted techniques provide interesting
                   opportunities to recognize fundamental trends and to
                   verify hypotheses.},
    doi          = {10.1016/j.anr.2016.04.002},
    isbn         = {0278-4319},
    issn         = {20937482},
    keywords     = {depression,kidney diseases,meta-analysis,music analysis
                   with computers,quality of life,self-management},
    mendeley-tags= {music analysis with computers},
    pmid         = {28057311},
    school       = {Technische Universit{\"{a}}t Ilmenau},
    type         = {Ph.D. Dissertation},
    url          = {https://www.db-thueringen.de/servlets/MCRFileNodeServlet/dbt_derivate_00039054/ilm1-2017000293.pdf}
}

Downloads: 0

{"_id":"S2ACNMpoPxErjippK","bibbaseid":"wei-computationalmethodsfortonalitybasedstyleanalysisofclassicalmusicaudiorecordings-2016","authorIDs":[],"author_short":["Weiß, C."],"bibdata":{"bibtype":"phdthesis","type":"Ph.D. Dissertation","author":[{"propositions":[],"lastnames":["Weiß"],"firstnames":["Christof"],"suffixes":[]}],"year":"2016","title":"Computational Methods for Tonality-Based Style Analysis of Classical Music Audio Recordings","abstract":"With the tremendously growing impact of digital technology, the ways of accessing music crucially changed. Nowadays, streaming services, download platforms, and private archives provide a large amount of music recordings to listeners. As tools for organizing and browsing such collections, automatic methods have become important. In the area of Music Informa- tion Retrieval, researchers are developing algorithms for analyzing and comparing music data with respect to musical characteristics. One typical application scenario is the classification of music recordings according to categories such as musical genres. In this thesis, we approach such classification problems with the goal of discriminating subgenres within Western classical music. In particular, we focus on typical categories such as historical periods or individual composers. From a musicological point of view, this classi- fication problem relates to the question of musical style, which constitutes a rather ill-defined and abstract concept. Usually, musicologists analyze musical scores in a manual fashion in order to acquire knowledge about style and its determining factors. This thesis contributes with computational methods for realizing such analyses on comprehensive corpora of audio recordings. Though it is hard to extract explicit information such as note events from audio data, the computational analysis of audio recordings might bear great potential for musi- cological research. One reason for this is the limited availability of symbolic scores in high quality. The style analysis experiments presented in this thesis focus on the fields of harmony and tonality. In the first step, we use signal processing techniques for computing chroma representations of the audio data. These semantic “mid-level” representations capture the pitch class content of an audio recording in a robust way and, thus, constitute a suitable starting point for subsequent processing steps. From such chroma representations, we derive measures for quantitatively describing stylistic properties of the music. Since chroma features suppress timbral characteristics to a certain extent, we hope to achieve invariance to timbre and instrumentation for our analysis methods. Inspired by the characteristics of the chroma representations, we model in this thesis specific concepts from music theory and propose algorithms to measure the occurence of certain tonal structures in audio recordings. One of the proposed methods aims at estimating the global key of a piece by considering the particular role of the final chord. Another contribution of this thesis is an automatic method to visualize modulations regarding diatonic scales as well as scale types over the course of a piece. Furthermore, we propose novel techniques for estimating the presence of specific interval and chord types and for measuring more abstract notions such as tonal complexity. In first experiments, we show the features' behavior for individual pieces and discuss their musical meaning. On the basis of these novel types of audio features, we perform comprehensive experiments for analyzing and classifying audio recordings regarding musical style. For this purpose, we apply methods from the field of machine learning. Using unsupervised clustering methods, we investigate the similarity of musical works across composers and composition years. Even though the underlying feature representations may be imprecise and error-prone in some cases, we can observe interesting tendencies that may exhibit some musical meaning when analyzing large databases. For example, we observe an increase of tonal complexity during the 19th and 20th century on the basis of our features. As an essential contribution of this dissertation, we perform automatic classification experiments according to historical periods (“eras”) and composers. We compile two datasets, on which we test common classifiers using both our tonal features and standardized audio features. Despite the vagueness of the task and the complexity of the data, we obtain good results for the classification with respect to historical periods. This indicates that the tonal features proposed in this thesis seem to robustly capture some stylistic properties. In contrast, using standardized timbral features for classification often leads to overfitting to the training data resulting in worse performance. Comparing different types of tonal features revealed that features relating to interval types, tonal complexity, and chord progressions are useful for classifying audio recordings with respect to musical style. This seems to validate the hypothesis that tonal characteristics can be discriminative for style analysis and that we can measure such characteristics directly from audio recordings. In summary, the interplay between musicology and audio signal processing can be very promising. When applied to a specific example, we have to be careful with the results of computational methods, which, of course, cannot compete with the experienced judgement of a musicologist. For analyzing comprehensive corpora, however, computer-assisted techniques provide interesting opportunities to recognize fundamental trends and to verify hypotheses.","doi":"10.1016/j.anr.2016.04.002","isbn":"0278-4319","issn":"20937482","keywords":"depression,kidney diseases,meta-analysis,music analysis with computers,quality of life,self-management","mendeley-tags":"music analysis with computers","pmid":"28057311","school":"Technische Universität Ilmenau","url":"https://www.db-thueringen.de/servlets/MCRFileNodeServlet/dbt_derivate_00039054/ilm1-2017000293.pdf","bibtex":"@PhDThesis{ wei2016-computational,\n author = {Wei{\\ss}, Christof},\n year = {2016},\n title = {Computational Methods for Tonality-Based Style Analysis\n of Classical Music Audio Recordings},\n abstract = {With the tremendously growing impact of digital\n technology, the ways of accessing music crucially changed.\n Nowadays, streaming services, download platforms, and\n private archives provide a large amount of music\n recordings to listeners. As tools for organizing and\n browsing such collections, automatic methods have become\n important. In the area of Music Informa- tion Retrieval,\n researchers are developing algorithms for analyzing and\n comparing music data with respect to musical\n characteristics. One typical application scenario is the\n classification of music recordings according to categories\n such as musical genres. In this thesis, we approach such\n classification problems with the goal of discriminating\n subgenres within Western classical music. In particular,\n we focus on typical categories such as historical periods\n or individual composers. From a musicological point of\n view, this classi- fication problem relates to the\n question of musical style, which constitutes a rather\n ill-defined and abstract concept. Usually, musicologists\n analyze musical scores in a manual fashion in order to\n acquire knowledge about style and its determining factors.\n This thesis contributes with computational methods for\n realizing such analyses on comprehensive corpora of audio\n recordings. Though it is hard to extract explicit\n information such as note events from audio data, the\n computational analysis of audio recordings might bear\n great potential for musi- cological research. One reason\n for this is the limited availability of symbolic scores in\n high quality. The style analysis experiments presented in\n this thesis focus on the fields of harmony and tonality.\n In the first step, we use signal processing techniques for\n computing chroma representations of the audio data. These\n semantic “mid-level” representations capture the pitch\n class content of an audio recording in a robust way and,\n thus, constitute a suitable starting point for subsequent\n processing steps. From such chroma representations, we\n derive measures for quantitatively describing stylistic\n properties of the music. Since chroma features suppress\n timbral characteristics to a certain extent, we hope to\n achieve invariance to timbre and instrumentation for our\n analysis methods. Inspired by the characteristics of the\n chroma representations, we model in this thesis specific\n concepts from music theory and propose algorithms to\n measure the occurence of certain tonal structures in audio\n recordings. One of the proposed methods aims at estimating\n the global key of a piece by considering the particular\n role of the final chord. Another contribution of this\n thesis is an automatic method to visualize modulations\n regarding diatonic scales as well as scale types over the\n course of a piece. Furthermore, we propose novel\n techniques for estimating the presence of specific\n interval and chord types and for measuring more abstract\n notions such as tonal complexity. In first experiments, we\n show the features' behavior for individual pieces and\n discuss their musical meaning. On the basis of these novel\n types of audio features, we perform comprehensive\n experiments for analyzing and classifying audio recordings\n regarding musical style. For this purpose, we apply\n methods from the field of machine learning. Using\n unsupervised clustering methods, we investigate the\n similarity of musical works across composers and\n composition years. Even though the underlying feature\n representations may be imprecise and error-prone in some\n cases, we can observe interesting tendencies that may\n exhibit some musical meaning when analyzing large\n databases. For example, we observe an increase of tonal\n complexity during the 19th and 20th century on the basis\n of our features. As an essential contribution of this\n dissertation, we perform automatic classification\n experiments according to historical periods (“eras”)\n and composers. We compile two datasets, on which we test\n common classifiers using both our tonal features and\n standardized audio features. Despite the vagueness of the\n task and the complexity of the data, we obtain good\n results for the classification with respect to historical\n periods. This indicates that the tonal features proposed\n in this thesis seem to robustly capture some stylistic\n properties. In contrast, using standardized timbral\n features for classification often leads to overfitting to\n the training data resulting in worse performance.\n Comparing different types of tonal features revealed that\n features relating to interval types, tonal complexity, and\n chord progressions are useful for classifying audio\n recordings with respect to musical style. This seems to\n validate the hypothesis that tonal characteristics can be\n discriminative for style analysis and that we can measure\n such characteristics directly from audio recordings. In\n summary, the interplay between musicology and audio signal\n processing can be very promising. When applied to a\n specific example, we have to be careful with the results\n of computational methods, which, of course, cannot compete\n with the experienced judgement of a musicologist. For\n analyzing comprehensive corpora, however,\n computer-assisted techniques provide interesting\n opportunities to recognize fundamental trends and to\n verify hypotheses.},\n doi = {10.1016/j.anr.2016.04.002},\n isbn = {0278-4319},\n issn = {20937482},\n keywords = {depression,kidney diseases,meta-analysis,music analysis\n with computers,quality of life,self-management},\n mendeley-tags= {music analysis with computers},\n pmid = {28057311},\n school = {Technische Universit{\\\"{a}}t Ilmenau},\n type = {Ph.D. Dissertation},\n url = {https://www.db-thueringen.de/servlets/MCRFileNodeServlet/dbt_derivate_00039054/ilm1-2017000293.pdf}\n}\n\n","author_short":["Weiß, C."],"key":"wei2016-computational","id":"wei2016-computational","bibbaseid":"wei-computationalmethodsfortonalitybasedstyleanalysisofclassicalmusicaudiorecordings-2016","role":"author","urls":{"Paper":"https://www.db-thueringen.de/servlets/MCRFileNodeServlet/dbt_derivate_00039054/ilm1-2017000293.pdf"},"keyword":["depression","kidney diseases","meta-analysis","music analysis with computers","quality of life","self-management"],"metadata":{"authorlinks":{}},"downloads":0},"bibtype":"phdthesis","biburl":"https://hmb.sampaio.me/bibliografia.bib.txt","creationDate":"2019-04-22T13:04:36.137Z","downloads":0,"keywords":["depression","kidney diseases","meta-analysis","music analysis with computers","quality of life","self-management"],"search_terms":["computational","methods","tonality","based","style","analysis","classical","music","audio","recordings","weiß"],"title":"Computational Methods for Tonality-Based Style Analysis of Classical Music Audio Recordings","year":2016,"dataSources":["n6MFY2CscQLDpJ7nT","RFLDZw5KyJdadDXDm"]}