Gaussian Processes For Machine – Carl Edward Rasmussen (1)

Work on string kernels was started by Watkins [1999] and Haussler [1999]. There are many further developments of the methods we have described above; for example Lodhi et al. [2001] go beyond substrings to consider subsequences of x which are not necessarily contiguous, and Leslie et al.

[2003] describe mismatch string kernels which allow substrings s and s′ of x and x′ respectively to match if there are at most m mismatches between them. We expect further developments in this area, tailoring (or engineering) the string kernels to have properties that make sense in a particular domain. The idea of string kernels, where we consider matches of substrings, can easily be extended to trees, e.g. by looking at matches of subtrees [Collins and Duffy, 2002].

Leslie et al. [2003] have applied string kernels to the classification of protein domains into SCOP12 superfamilies. The results obtained were significantly better than methods based on either PSI-BLAST13 searches or a generative hidden Markov model classifier. Similar results were obtained by Jaakkola et al. [2000] using a Fisher kernel (described in the next section).

Saunders et al. [2003] have also described the use of string kernels on the problem of classifying natural language newswire stories from the Reuters-2157814 database into ten classes. 4.4.2 Fisher Kernels As explained above, our problem is that the input x is a structured object of arbitrary size e.g. a string, and we wish to extract features from it. The Fisher kernel (introduced by Jaakkola et al., 2000) does this by taking a generative model p(x|θ), where θ is a vector of parameters, and computing the feature vector φθ(x) = ∇θ log p(x|θ).

φθ(x) is sometimes called the score vector. score vector Take, for example, a Markov model for strings. Let xk be the kth symbol in string x. Then a Markov model gives p(x|θ) = p(x1|π)Q|x|−1 i=1 p(xi+1|xi, A), where θ = (π, A). Here (π)j gives the probability that x1 will be the jth symbol in the alphabet A, and A is a |A| × |A| stochastic matrix, with ajk giving the probability that p(xi+1 = k|xi = j). Given such a model it is straightforward to compute the score vector for a given x.

It is also possible to consider other generative models p(x|θ). For example we might try a kth-order Markov model where xi is predicted by the preceding k symbols. See Leslie et al. [2003] and Saunders et al.

Carl Edward Rasmussen and Christopher K. I. Williams Gaussian Processes for Machine Learning Adaptive Computation and Machine Learning Thomas Dietterich, Editor Christopher Bishop, David Heckerman, Michael Jordan, and Michael Kearns, Associate Editors Bioinformatics: The Machine Learning Approach, Pierre Baldi and Søren Brunak Reinforcement Learning: An Introduction, Richard S. Sutton and Andrew G. Barto Graphical Models for Machine Learning and Digital Communication, Brendan J.

Frey Learning in Graphical Models, Michael I. Jordan Causation, Prediction, and Search, second edition, Peter Spirtes, Clark Glymour, and Richard Scheines Principles of Data Mining, David Hand, Heikki Mannila, and Padhraic Smyth Bioinformatics: The Machine Learning Approach, second edition, Pierre Baldi and Søren Brunak Learning Kernel Classifiers: Theory and Algorithms, Ralf Herbrich Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, Bernhard Sch¨olkopf and Alexander J.

Smola Introduction to Machine Learning, Ethem Alpaydin Gaussian Processes for Machine Learning, Carl Edward Rasmussen and Christopher K. I. Williams Gaussian Processes for Machine Learning Carl Edward Rasmussen Christopher K. I. Williams The MIT Press Cambridge, Massachusetts London, England c⃝2006 Massachusetts Institute of Technology All rights reserved.

No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher. MIT Press books may be purchased at special quantity discounts for business or sales promotional use. For information, please email special [email protected] or write to Special Sales Department, The MIT Press, 55 Hayward Street, Cambridge, MA 02142.

Typeset by the authors using LATEX2ε. This book printed and bound in the United States of America. Library of Congress Cataloging-in-Publication Data Rasmussen, Carl Edward. Gaussian processes for machine learning / Carl Edward Rasmussen, Christopher K. I. Williams. p. cm. —(Adaptive computation and machine learning) Includes bibliographical references and indexes. ISBN 0-262-18253-X 1. Gaussian processes—Data processing. 2. Machine learning—Mathematical models. I. Williams, Christopher K. I. II. Title. III. Series.

QA274.4.R37 2006 519.2’3—dc22 2005053433 10 9 8 7 6 5 4 3 2 1 The actual science of logic is conversant at present only with things either certain, impossible, or entirely doubtful, none of which (fortunately) we have to reason on. Therefore the true logic for this world is the calculus of Probabilities, which takes account of the magnitude of the probability which is, or ought to be, in a reasonable man’s mind. — James Clerk Maxwell [1850] Contents Series Foreword . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Preface . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . .

This is a short excerpt from the opening of “” by Unknown, quoted for review and introduction purposes. All rights belong to the copyright holders.

Book Information

• Unique ID: 01d17c93d77919b2
• File Extension: .pdf
• File Size: 2,998,290 bytes (2.859 MB)
• Title: –
• Author: Unknown
• ISBN: 026218253X
• Pages: 267
• Language: English (en)

Reading & Word Statistics

• Estimated Reading Time: 553.05 minutes
• Total Words: 110,611
• Total Characters: 634,097
• Average Words per Page: 414.27
• Average Characters per Page: 2374.9

Most Frequent Words

function (695), see (527), covariance (515), gaussian (508), process (440), log (400), functions (368), model (342), likelihood (330), section (319), using (283), one (283), kernel (271), regression (246), matrix (242), data (240), marginal (232), also (225), classification (218), mean (217), given (213), set (211), distribution (191), training (189), case (184), input (184), test (178), linear (177), approximation (173), predictive (172), learning (171), posterior (171), figure (169), points (167), vector (160), example (159), methods (157), models (154), noise (154), problem (149), two (149), use (148), prior (147), method (143), used (139), processes (134), obtain (130), probability (129), exp (128), variance (128), information (126), thus (126), between (124), space (123), hyperparameters (123), consider (123), predictions (122), however (122), parameters (120), number (116), note (115), values (114), form (110), error (105), neural (103), latent (102), machine (101), squared (101), loss (100), bayesian (99), obtained (98), analysis (97), panel (97), prediction (94), class (94), point (94), corresponding (93), random (92), first (89), term (88), let (86), shows (85), algorithm (84), binary (82), selection (82), value (82), notice (82), chapter (81), length-scale (81), problems (80), gives (79), press (78), equivalent (78), terms (77), different (77), show (76), compute (76), general (75), large (74), although (74).

PDF Download