Thursday, June 16, 2011
Conference paper deadline: CIDM 2011
The deadline for papers submitted to the Second International Workshop on Computational Intelligence for Disaster Management (CIDM-2011) is 30 June, 2011. This conference will be held in Fukuoka, Japan, November 30 - December 2, 2011.
Wednesday, June 15, 2011
Call for papers: FUZZ-IEEE 2012
The deadline for papers submitted to the IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) is December 19, 2011. This conference is part of the 2012 IEEE World Congress on Computational Intelligence (WCCI 2012) and is held concurrently with the 2012 Congress on Evolutionary Computation (CEC) and the International Joint Conference on Neural Networks (IJCNN). WCCI 2012 will be held in Brisbane, Australia, June 10-15, 2012.
Conference paper deadline: ACAL 11
The deadline for papers submitted to the 5th Australian Conference on Artificial Life (ACAL) 2011 is 28 June 2011. This conference will be held in Perth, Australia, 6-8 December, 2011.
Paper submission deadline: UKCI 2011
The deadline for papers submitted to the 11th UK Workshop on Computational Intelligence (UKCI) 2011 is 20 June 2011. This workshop will be held in Manchester, UK, 7-9 September, 2011.
Paper submission deadline: ACIIDS 2012
The deadline for papers submitted to the 4th Asian Conference on Intelligent Information and Database Systems (ACIIDS) 2012 is September 15, 2011. This conference will be held in Kaohsiung, Taiwan, March 19-21, 2012.
Tuesday, June 14, 2011
Detecting reefs with ANN
I have just published a paper, along with several of my colleagues at the University of Adelaide, on detecting reefs using MLP.
The problem was that while there is coarse-scale bathymetric data from sonar surveys, and surveys of small areas that list the presence and absence of reefs in a relatively small number of points, there have not been large-scale surveys of where, exactly, reefs are. This is because the fine-scale sonar surveys needed to detect them remotely are very expensive and time consuming, and surveying manually (divers going into the water and looking) can be dangerous in places (either dangerous sea conditions, or big bitey beasties in the water). Not knowing where reefs are is a problem, especially if you want to construct ecological models of reef-dwelling creatures like abalone. In short, abalone like to live on reefs, so to build an accurate model, you must know where the reefs are.
We addressed this problem by firstly, processing the bathymetric data into slope and curvature measures of the sea bed, then training MLP over sliding 2D windows of these variables, where a known reef presence or absence was in the centre of the window. A window in this case was an n * n matrix of values, where we used n=5. So, the third element of the third row was the target cell, which the MLP was learning to classify as either a reef or non-reef point.
We found that combinations of the bathymetric value of the target cell, and a 5*5 window of seabed slope, gave us the best results. The overall experimental method we used was as I described in this post. While we weren't able to classify every reef exactly, the overall accuracy of 85% was enough to construct a useful map of reefs for ecological models of abalone.
We're looking at boosting the accuracy of our models by various means - this first paper is just a proof-of-concept, to show that we can find reefs with ANN.
The full citation for this paper is:
Watts, M.J., et al., A novel method for mapping reefs and subtidal rocky habitats using artificial neural networks. Ecological Modelling (2011), doi:10.1016/j.ecolmodel.2011.04.024
The problem was that while there is coarse-scale bathymetric data from sonar surveys, and surveys of small areas that list the presence and absence of reefs in a relatively small number of points, there have not been large-scale surveys of where, exactly, reefs are. This is because the fine-scale sonar surveys needed to detect them remotely are very expensive and time consuming, and surveying manually (divers going into the water and looking) can be dangerous in places (either dangerous sea conditions, or big bitey beasties in the water). Not knowing where reefs are is a problem, especially if you want to construct ecological models of reef-dwelling creatures like abalone. In short, abalone like to live on reefs, so to build an accurate model, you must know where the reefs are.
We addressed this problem by firstly, processing the bathymetric data into slope and curvature measures of the sea bed, then training MLP over sliding 2D windows of these variables, where a known reef presence or absence was in the centre of the window. A window in this case was an n * n matrix of values, where we used n=5. So, the third element of the third row was the target cell, which the MLP was learning to classify as either a reef or non-reef point.
We found that combinations of the bathymetric value of the target cell, and a 5*5 window of seabed slope, gave us the best results. The overall experimental method we used was as I described in this post. While we weren't able to classify every reef exactly, the overall accuracy of 85% was enough to construct a useful map of reefs for ecological models of abalone.
We're looking at boosting the accuracy of our models by various means - this first paper is just a proof-of-concept, to show that we can find reefs with ANN.
The full citation for this paper is:
Watts, M.J., et al., A novel method for mapping reefs and subtidal rocky habitats using artificial neural networks. Ecological Modelling (2011), doi:10.1016/j.ecolmodel.2011.04.024
Call for papers: IJCNN 2012
The deadline for papers submitted to the International Joint Conference on Neural Networks is December 19, 2011. This conference is part of the 2012 IEEE World Congress on Computational Intelligence (WCCI 2012) and is held concurrently with the 2012 Congress on Evolutionary Computation (CEC) and IEEE Conference on Fuzzy Logic (Fuzz-IEEE). WCCI 2012 will be held in Brisbane, Australia, June 10-15, 2012.
Monday, June 13, 2011
IJCNN 2011 Final Program
The final program for the 2011 International Joint Conference on Neural Networks (IJCNN) has been posted. This conference will be held in San Jose, California, July 31 - August 5, 2011.
Saturday, June 11, 2011
Call for papers: CEC 2012
The deadline for papers submitted to the 2012 Congress on Evolutionary Computation is December 19, 2011. This conference is part of the 2012 IEEE World Congress on Computational Intelligence (WCCI 2012) and is held concurrently with the 2012 International Joint Conference on Neural Networks (IJCNN) and IEEE Conference on Fuzzy Logic (Fuzz-IEEE). WCCI 2012 will be held in Brisbane, Australia, June 10-15, 2012.
Friday, June 10, 2011
Conference paper deadline: CIBCB 2012
The deadline for papers submitted to the 2012 conference on Computational Intelligence in Bioinformatics and Computational Biology is November 20, 2011. This conference will be held in San Diego, California, May 9-12, 2012.
Tuesday, May 31, 2011
Social Internetworking
I've just put online a brief description (PDF) of how I connect this blog to other social media sites, including LinkedIn, Twitter, and ResearchGATE.
I describe it as "Social Internetworking" (not an original phrase) and it involves using various free aggregators and web services to export blog posts.
You can follow me on Twitter at https://twitter.com/#!/DrMikeWatts, where you will see updates to this blog as soon as they are posted. You can find a complete list of my social networking profiles on my personal web page, mike.watts.net.nz.
I describe it as "Social Internetworking" (not an original phrase) and it involves using various free aggregators and web services to export blog posts.
You can follow me on Twitter at https://twitter.com/#!/DrMikeWatts, where you will see updates to this blog as soon as they are posted. You can find a complete list of my social networking profiles on my personal web page, mike.watts.net.nz.
Saturday, May 28, 2011
Fuzzy Markup Language
Giovanni Acampora describes the Fuzzy Markup Language (FML) in a series of articles. FML is a XML-based method for describing fuzzy logic systems. Fields in the schema specify the fuzzy knowledge base, which consists of the fuzzy variables and their membership functions, and the fuzzy rule base. The schema also allows for the specification of the inference and defuzzification method to use, and the type of fuzzy system (Zadeh-Mamdani or Takagi-Sugeno-Kang). Finally, it supports distributed fuzzy rule systems, that is, the user can specify the IP address of machines on which parts of the fuzzy system should run.
The major advantage of using XML to describe a fuzzy system is interoperability. All that is needed to read an XML file is the appropriate schema for that file, and an XML parser. This makes it much easier to exchange fuzzy systems between software: for example, an application could extract fuzzy rules from a neural network (like the EFuNN and SECoS rule extraction algorithms that exist) which could then be read directly into a fuzzy inference engine or uploaded into a fuzzy controller. Also, with technologies like XSLT, it is possible to compile the FML into the programming language of your choice, ready for embedding into whatever application you please.
Although Acampora's motivation for developing FML seems to be to develop embedded fuzzy controllers for ambient intelligence applications, FML could be a real boon for developers of fuzzy rule extraction algorithms: from my own experience during my PhD, I know that having to design a file format and implement the appropriate parsers for rule extraction and fuzzy inference engines can be a real pain, taking as much time as implementing the rule extraction algorithm itself. I would much rather have used something like FML for my work.
Such standard, XML-based file formats would be useful for other areas of computational intelligence: a standard XML format for ANN, for example, would be fairly simple to implement and also very useful. I could imagine, for example, training a MLP, saving it in an XML-based format, then using XSLT to transform it to C++ and uploading it into an embedded controller. Conventional, static-architecture ANN like perceptrons, MLP, or SOM could easily be represented in XML.
I will be watching for further developments in this area of technology: I've had quite enough of designing my own file formats!
The major advantage of using XML to describe a fuzzy system is interoperability. All that is needed to read an XML file is the appropriate schema for that file, and an XML parser. This makes it much easier to exchange fuzzy systems between software: for example, an application could extract fuzzy rules from a neural network (like the EFuNN and SECoS rule extraction algorithms that exist) which could then be read directly into a fuzzy inference engine or uploaded into a fuzzy controller. Also, with technologies like XSLT, it is possible to compile the FML into the programming language of your choice, ready for embedding into whatever application you please.
Although Acampora's motivation for developing FML seems to be to develop embedded fuzzy controllers for ambient intelligence applications, FML could be a real boon for developers of fuzzy rule extraction algorithms: from my own experience during my PhD, I know that having to design a file format and implement the appropriate parsers for rule extraction and fuzzy inference engines can be a real pain, taking as much time as implementing the rule extraction algorithm itself. I would much rather have used something like FML for my work.
Such standard, XML-based file formats would be useful for other areas of computational intelligence: a standard XML format for ANN, for example, would be fairly simple to implement and also very useful. I could imagine, for example, training a MLP, saving it in an XML-based format, then using XSLT to transform it to C++ and uploading it into an embedded controller. Conventional, static-architecture ANN like perceptrons, MLP, or SOM could easily be represented in XML.
I will be watching for further developments in this area of technology: I've had quite enough of designing my own file formats!
Tuesday, May 24, 2011
Evolving Connectionist Systems
An interesting family of neural networks is Evolving Connectionist Systems (ECoS). These were invented by Professor Nik Kasabov around 1998. ECoS are constructive networks, that is, they do not start with a fixed structure but instead grow (add neurons) as training data is presented to them. The advantages of this are:
The first ECoS was the Evolving Fuzzy Neural Network EFuNN. Later ECoS include the Simple Evolving Connectionist System SECoS (which is really an EFuNN with the fuzzy logic elements removed) and the Evolving Clustering Method ECM. EFuNN and SECoS both have rule extraction algorithms associated with them, by which fuzzy rules can be extracted from a trained EFuNN or SECoS network. This makes ECoS very useful for data mining, especially in an online application area.
I wrote a review of ECoS technology a couple of years ago, in this paper. An online reprint is available here. I also maintain a website of resources on ECoS networks at: ecos.watts.net.nz.
Research on ECoS networks is continuing, especially at Prof. Kasabov's lab KEDRI. Nowadays, ECoS research is focused on spiking neuron models, that is, neurons that include a temporal aspect to their activation, much as biological neurons do.
- they are fast learning, as they learn the data as it presented, rather than iteratively
- they are hard to over-train, as new data is accommodated by adding new neurons to the network
The first ECoS was the Evolving Fuzzy Neural Network EFuNN. Later ECoS include the Simple Evolving Connectionist System SECoS (which is really an EFuNN with the fuzzy logic elements removed) and the Evolving Clustering Method ECM. EFuNN and SECoS both have rule extraction algorithms associated with them, by which fuzzy rules can be extracted from a trained EFuNN or SECoS network. This makes ECoS very useful for data mining, especially in an online application area.
I wrote a review of ECoS technology a couple of years ago, in this paper. An online reprint is available here. I also maintain a website of resources on ECoS networks at: ecos.watts.net.nz.
Research on ECoS networks is continuing, especially at Prof. Kasabov's lab KEDRI. Nowadays, ECoS research is focused on spiking neuron models, that is, neurons that include a temporal aspect to their activation, much as biological neurons do.
Wednesday, May 18, 2011
Modelling distribution of jellyfish with ANN
A new paper first-authored by David Pontin, my ex-PhD student from Lincoln University. This describes how he used MLP to model to presence and absence of a species of stinging jellyfish (Physalia physalis) at New Zealand beaches.
There are a couple of interesting points about this paper. Firstly, because there have been no surveys of Physalia distribution, a surrogate data set was used. This data set was stings recorded by lifeguards of Surf Lifesaving New Zealand. Since lifeguards treat jellyfish stings, each incident has to be recorded, and Physalia is the only stinging organism in New Zealand waters, a fairly large data set was available as to the presence of these jellyfish. Predictions were made from oceanic variables such as wave height and direction, and wind speed and direction.
Secondly, the data was carefully cleaned: since stings of swimmers was used as the surrogate for Physalia presence, times when there were no swimmers at the beach were excluded from the data set. While this introduced a small missing-not-at-random bias, it also removed a large number of false absences: if an example was recorded as an absence, then it was because there were no stings recorded, not because there was no one in the water.
Thirdly, an analysis of the contributions of each input of the ANN was performed. This showed which of the oceanic variables contributed the most to the presence of Physalia. This analysis indicated that there may be a hitherto unknown spawning ground for this species in the Tasman Sea.
Finally, and this is in many ways the focus of the paper, the contribution analysis of the ANN was compared with the results of input contribution analysis by an evolutionary algorithm.
Overall, this is a nice little paper that neatly sums up David's work and contributes to the understanding of the behaviour of Physalia. This shows how useful computational intelligence is to ecological applications, an area where there is, in my opinion, enormous potential for computational intelligence researchers to make real, meaningful contributions.
There are a couple of interesting points about this paper. Firstly, because there have been no surveys of Physalia distribution, a surrogate data set was used. This data set was stings recorded by lifeguards of Surf Lifesaving New Zealand. Since lifeguards treat jellyfish stings, each incident has to be recorded, and Physalia is the only stinging organism in New Zealand waters, a fairly large data set was available as to the presence of these jellyfish. Predictions were made from oceanic variables such as wave height and direction, and wind speed and direction.
Secondly, the data was carefully cleaned: since stings of swimmers was used as the surrogate for Physalia presence, times when there were no swimmers at the beach were excluded from the data set. While this introduced a small missing-not-at-random bias, it also removed a large number of false absences: if an example was recorded as an absence, then it was because there were no stings recorded, not because there was no one in the water.
Thirdly, an analysis of the contributions of each input of the ANN was performed. This showed which of the oceanic variables contributed the most to the presence of Physalia. This analysis indicated that there may be a hitherto unknown spawning ground for this species in the Tasman Sea.
Finally, and this is in many ways the focus of the paper, the contribution analysis of the ANN was compared with the results of input contribution analysis by an evolutionary algorithm.
Overall, this is a nice little paper that neatly sums up David's work and contributes to the understanding of the behaviour of Physalia. This shows how useful computational intelligence is to ecological applications, an area where there is, in my opinion, enormous potential for computational intelligence researchers to make real, meaningful contributions.
Monday, May 16, 2011
Minimum Requirements for Computational Intelligence Papers
I am reposting this after it was lost during the Blogger meltdown last week.
In a previous post, I mentioned some challenges in reviewing computational intelligence papers. In this post, I list what I consider to be the minimum requirements for computational intelligence papers. These are the things that I look for when I review a paper, and if they aren't there, I reject it.
1. Define all variables in equations
While most computational intelligence papers have mathematics in them, a disappointingly large number of them do not define the variables in their equations. Or, if they do, they define them some distance from the equation itself. If I am reading your paper, I want to understand the maths, and I can't do that if I can't quickly find the meaning of each variable.
2. Use more than one data set to test an algorithm
If your paper describes a new algorithm, or even an improvement on an existing algorithm, it must be tested on more than one data set. The No Free Lunch theorem tells us that there are always some data sets on which every algorithm will perform well, and some on which it will perform poorly. While publication bias means that poor results often do not get reported, I do expect results over more than one data set.
3. Investigate more than one set of parameters
For any algorithm, there will be one set of parameters that yields better performance than others. This means that if your study only utilised a single set of parameters, you cannot tell whether you might have gotten better results using different parameters. For new algorithms, it is useful to show how sensitive the performance of the algorithm is to its parameters.
4. Clearly describe how the parameters were chosen
This is a particular problem with papers that describe applications of algorithms. In short, even if you only list the parameters that gave the best performance, you should still describe how you chose them. Choosing parameters by trial-and-error is fine, but you must say in the paper that that was how you chose your parameters. Also, LIST THE PARAMETERS IN THE PAPER! Being able to replicate experiments is at the very heart of science, and if you don't say what your parameters were, your experiments can't be replicated.
5. Use multiple partitions of the data set
Neural network papers are particularly bad for this. Often, the algorithm will be trained on one subset of the data (the training set) then tested on the remaining data (the testing or validation set, depending on who's writing the paper). Sometimes the data is divided into subsets randomly, sometimes it is not. There are two problems with this approach: firstly, it is entirely possible that the data is partitioned in a way that is particularly good for the algorithm, that is, the reported performance of the algorithm is due to the partitioning of the data, rather than the algorithm itself; Secondly, if the training parameters of the algorithm are chosen to maximise the performance over the testing set, that is equivalent to training over the testing set: that is, the testing set is no longer independent.
A better way (and my preferred technique) is to use k-fold cross-validation with an independent validation data set. In this technique, a validation data set is either randomly extracted or sourced separately from the training data set. The training data set is then divided into k-subsets, and the algorithm trained over k-1 of the subsets. The kth subset is then used to evaluate the performance of the algorithm. This is repeated k times, with a different subset held out as the evaluation set each time. This has the effect of training and testing over the entire data set. The results over the cross-validation are used to select the parameters, and the final performance is assessed over the validation data set. Since the validation data set is not used to select the parameters or to train the algorithm, it remains statistically independent.
6. The final testing / validation set must be independent
This means that the data in the validation set must be from a separate process to that which produced the data used for the k-fold cross-validation. If you are training an ANN to recognise speech, the validation set should be from a different speaker to those it was trained on, or at least from a different recording. If you are training an ANN to recognise spatial features, the validation data should come from a different area or different survey to the data that was used to train the ANN.
7. Compare a new algorithm with an existing algorithm
If you are claiming that your new algorithm work well and is highly accurate, then you need to prove that by comparing it against the performance of an existing, preferably well-known, algorithm. You don't need to perform the experiments with the existing algorithm yourself (although it is good if you do), you can point to previously published results. But a comparison must be carried out.
8. Comparisons of performance must be done in a statistically sound manner
This means that you can't just look at two numbers (two means) and say that your algorithm is better because the mean accuracy is higher than that of an existing algorithm. Comparisons must be done using statistical tests, that is, I want to know whether the results are significantly different. If you say that the results are significantly different, then you must also specify what statistical test was performance. For example, it is best to say something along the lines of "the accuracy of the Bogon 2000 algorithm was significantly higher than that of the Wibble 12 algorithm (two-tailed t-test, p=0.01)".
If you don't want to follow these principles, that's fine, as long as you explain in your paper (or review rejoinder) why you didn't do that. I'm quite prepared to be shown to be wrong.
In a previous post, I mentioned some challenges in reviewing computational intelligence papers. In this post, I list what I consider to be the minimum requirements for computational intelligence papers. These are the things that I look for when I review a paper, and if they aren't there, I reject it.
1. Define all variables in equations
While most computational intelligence papers have mathematics in them, a disappointingly large number of them do not define the variables in their equations. Or, if they do, they define them some distance from the equation itself. If I am reading your paper, I want to understand the maths, and I can't do that if I can't quickly find the meaning of each variable.
2. Use more than one data set to test an algorithm
If your paper describes a new algorithm, or even an improvement on an existing algorithm, it must be tested on more than one data set. The No Free Lunch theorem tells us that there are always some data sets on which every algorithm will perform well, and some on which it will perform poorly. While publication bias means that poor results often do not get reported, I do expect results over more than one data set.
3. Investigate more than one set of parameters
For any algorithm, there will be one set of parameters that yields better performance than others. This means that if your study only utilised a single set of parameters, you cannot tell whether you might have gotten better results using different parameters. For new algorithms, it is useful to show how sensitive the performance of the algorithm is to its parameters.
4. Clearly describe how the parameters were chosen
This is a particular problem with papers that describe applications of algorithms. In short, even if you only list the parameters that gave the best performance, you should still describe how you chose them. Choosing parameters by trial-and-error is fine, but you must say in the paper that that was how you chose your parameters. Also, LIST THE PARAMETERS IN THE PAPER! Being able to replicate experiments is at the very heart of science, and if you don't say what your parameters were, your experiments can't be replicated.
5. Use multiple partitions of the data set
Neural network papers are particularly bad for this. Often, the algorithm will be trained on one subset of the data (the training set) then tested on the remaining data (the testing or validation set, depending on who's writing the paper). Sometimes the data is divided into subsets randomly, sometimes it is not. There are two problems with this approach: firstly, it is entirely possible that the data is partitioned in a way that is particularly good for the algorithm, that is, the reported performance of the algorithm is due to the partitioning of the data, rather than the algorithm itself; Secondly, if the training parameters of the algorithm are chosen to maximise the performance over the testing set, that is equivalent to training over the testing set: that is, the testing set is no longer independent.
A better way (and my preferred technique) is to use k-fold cross-validation with an independent validation data set. In this technique, a validation data set is either randomly extracted or sourced separately from the training data set. The training data set is then divided into k-subsets, and the algorithm trained over k-1 of the subsets. The kth subset is then used to evaluate the performance of the algorithm. This is repeated k times, with a different subset held out as the evaluation set each time. This has the effect of training and testing over the entire data set. The results over the cross-validation are used to select the parameters, and the final performance is assessed over the validation data set. Since the validation data set is not used to select the parameters or to train the algorithm, it remains statistically independent.
6. The final testing / validation set must be independent
This means that the data in the validation set must be from a separate process to that which produced the data used for the k-fold cross-validation. If you are training an ANN to recognise speech, the validation set should be from a different speaker to those it was trained on, or at least from a different recording. If you are training an ANN to recognise spatial features, the validation data should come from a different area or different survey to the data that was used to train the ANN.
7. Compare a new algorithm with an existing algorithm
If you are claiming that your new algorithm work well and is highly accurate, then you need to prove that by comparing it against the performance of an existing, preferably well-known, algorithm. You don't need to perform the experiments with the existing algorithm yourself (although it is good if you do), you can point to previously published results. But a comparison must be carried out.
8. Comparisons of performance must be done in a statistically sound manner
This means that you can't just look at two numbers (two means) and say that your algorithm is better because the mean accuracy is higher than that of an existing algorithm. Comparisons must be done using statistical tests, that is, I want to know whether the results are significantly different. If you say that the results are significantly different, then you must also specify what statistical test was performance. For example, it is best to say something along the lines of "the accuracy of the Bogon 2000 algorithm was significantly higher than that of the Wibble 12 algorithm (two-tailed t-test, p=0.01)".
If you don't want to follow these principles, that's fine, as long as you explain in your paper (or review rejoinder) why you didn't do that. I'm quite prepared to be shown to be wrong.
Tuesday, May 10, 2011
Paper submission deadline: MLMI 2011
The deadline for papers submitted to the Second International Workshop on Machine Learning in Medical Imaging (MLMI 2o11) is June 1, 2011. This workshop will be held in Toronto, Canada, 18 September, 2011.
Monday, May 9, 2011
Paper submission deadline: AAIA'11
The deadline for submitting papers to the 6th International Symposium Advances in Artificial Intelligence and Applications (AAIA'11) is 31 May 2011. Symposium will be held in Szczecin, Poland, 19-21 September, 2011.
Sunday, May 8, 2011
Deadline for submission of extended abstracts: IEEE HST
The deadline for submission of extended abstracts to the 2011 IEEE Conference on Technologies for Homeland Security is May 13, 2011. If the abstract is accepted, a full paper must be submitted by June 24, 2011. This conference will be held in Greater Boston, 15-17 November, 2011.
Saturday, May 7, 2011
Submission deadline: EPIA 2011
The deadline for submission of abstracts to the XV Portuguese Conference on Artificial Intelligence is May 10 2011. This conference will be held in Lisbon, Portugal, 10-13 October, 2011.
Deadline extension: EA 2011
The deadline for the 2011 Artificial Evolution conference (EA2011) has been extended to 15 May 2011. This conference will be held in Angers, France, 24-26 October, 2011.
Subscribe to:
Posts (Atom)
Posts
